Hey.
You can use expon:
http://nbviewer.ipython.org/github/amueller/pydata-nyc-advanced-sklearn/blob/master/Chapter%203%20-%20Randomized%20Hyper%20Parameter%20Search.ipynb
(it has a scale parameter)
Andy
On 04/07/2015 05:16 PM, Arnaud Fouchet wrote:
Hello,
I have a little question about Random Search vs Grid Search. I am used
to grid search over parameter increasing exponentially (lambda = 0.01,
0.1, ... 1000). When you use randomized search, what type of
distribution do you use ? If I use an uniform distribution, I would
test hyper-parameter values that are in the same order of magnitude.
Thank you,
Arnaud
2015-04-07 21:28 GMT+02:00 Andreas Mueller <t3k...@gmail.com
<mailto:t3k...@gmail.com>>:
I talks about it a bunch here:
https://www.youtube.com/watch?v=0wUF_Ov8b0A
The paper is here: http://www.jmlr.org/papers/v13/bergstra12a.html
I'd be interested in the blog post.
Really I don't think searching over losses and regularization will
give you much usually.
I have not really seen any experienced machine learner do it.
On 04/07/2015 03:01 PM, Jason Wolosonovich wrote:
Hi Roberto,
I’m no expert by any means, but I was reading a blog post the
other day that talked about using Random Search vs Grid Search.
The gist of the article is that, since you can feed distributions
to Random Search and it selects values randomly over the number
of iterations you choose, it is a better initial choice when
you’re not sure which parameters/combinations to use (which is
usually my case J) and you’ll end up with Random Search finding
more useful parameters faster than if you tell Grid Search to
search over combinations (some of which may have no potential to
help you). Then when you see the results of the Random Search,
you can use that information to search a narrower range of
values/parameters (a finer grid) exhaustively using Grid Search.
Unfortunately, I thought I bookmarked the article but I can’t
find it. I’ll keep looking though and send it out if I do.
Additionally, the docs for the individual estimators in Sklearn
tell you what parameters are not valid with each other, so you
wouldn’t want to put those parameters together in your param_grid
dictionary. For your dictionary (as others have already
mentioned) just make sure that you only provide options in each
of your dictionaries that can be used together. You can pass a
list of dictionaries to param_grid like Sebastian just demonstrated.
Check the links below as well, Random Search comes up with just
about the same results as Grid Search, but faster/more
efficiently. Hope this helps.
Scikit Docs:
http://scikit-learn.org/stable/modules/grid_search.html#grid-search-tips
http://scikit-learn.org/stable/auto_examples/model_selection/randomized_search.html#example-model-selection-randomized-search-py
-Jason
*From:*Pagliari, Roberto [mailto:rpagli...@appcomsci.com]
*Sent:* Tuesday, April 07, 2015 9:24 AM
*To:* scikit-learn-general@lists.sourceforge.net
<mailto:scikit-learn-general@lists.sourceforge.net>
*Subject:* [Scikit-learn-general] CV with SVM
not all combinations of cost/loss functions and dual are possible
with SVM.
when performing grid search with CV, does sklearn skip invalid
combinations?
Thank you,
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
<mailto:Scikit-learn-general@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live
exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
<mailto:Scikit-learn-general@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
