I talks about it a bunch here:
https://www.youtube.com/watch?v=0wUF_Ov8b0A
The paper is here: http://www.jmlr.org/papers/v13/bergstra12a.html
I'd be interested in the blog post.
Really I don't think searching over losses and regularization will give
you much usually.
I have not really seen any experienced machine learner do it.
On 04/07/2015 03:01 PM, Jason Wolosonovich wrote:
Hi Roberto,
I’m no expert by any means, but I was reading a blog post the other
day that talked about using Random Search vs Grid Search. The gist of
the article is that, since you can feed distributions to Random Search
and it selects values randomly over the number of iterations you
choose, it is a better initial choice when you’re not sure which
parameters/combinations to use (which is usually my case J) and you’ll
end up with Random Search finding more useful parameters faster than
if you tell Grid Search to search over combinations (some of which may
have no potential to help you). Then when you see the results of the
Random Search, you can use that information to search a narrower range
of values/parameters (a finer grid) exhaustively using Grid Search.
Unfortunately, I thought I bookmarked the article but I can’t find it.
I’ll keep looking though and send it out if I do.
Additionally, the docs for the individual estimators in Sklearn tell
you what parameters are not valid with each other, so you wouldn’t
want to put those parameters together in your param_grid dictionary.
For your dictionary (as others have already mentioned) just make sure
that you only provide options in each of your dictionaries that can be
used together. You can pass a list of dictionaries to param_grid like
Sebastian just demonstrated.
Check the links below as well, Random Search comes up with just about
the same results as Grid Search, but faster/more efficiently. Hope
this helps.
Scikit Docs:
http://scikit-learn.org/stable/modules/grid_search.html#grid-search-tips
http://scikit-learn.org/stable/auto_examples/model_selection/randomized_search.html#example-model-selection-randomized-search-py
-Jason
*From:*Pagliari, Roberto [mailto:rpagli...@appcomsci.com]
*Sent:* Tuesday, April 07, 2015 9:24 AM
*To:* scikit-learn-general@lists.sourceforge.net
*Subject:* [Scikit-learn-general] CV with SVM
not all combinations of cost/loss functions and dual are possible with
SVM.
when performing grid search with CV, does sklearn skip invalid
combinations?
Thank you,
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
