On Wed, 23 Nov 2016 at 17:31 Joel Nothman <joel.noth...@gmail.com> wrote:
> Raghav's example of > > > [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': > 'sgd'}, {'solver': 'adam'}] > > was not correct. > > Should be > > > [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': > ['sgd']}, {'solver': ['adam']}] > > (Note all values of dicts are lists) > Ah, thanks! (Just ran into an error as it started iterating over the "sgd".) > > On 23 November 2016 at 22:52, Jaidev Deshpande <deshpande.jai...@gmail.com > > wrote: > > > > On Wed, 23 Nov 2016 at 16:29 Raghav R V <rag...@gmail.com> wrote: > > Hi! > > What you could do is specify lists of dicts to group the parameters which > apply together in one dict... > > [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': > 'sgd'}, {'solver': 'adam'}] > > ```py > from sklearn.neural_network import MLPClassifier > from sklearn.model_selection import GridSearchCV > from sklearn.datasets import make_classification > > from pandas import DataFrame > > X, y = make_classification(random_state=42) > > gs = GridSearchCV(MLPClassifier(random_state=42), > param_grid=[{'learning_rate': ['constant', 'invscaling', > 'adaptive'], > 'solver': ['sgd',]}, > {'solver': ['adam',]}]) > > DataFrame(gs.fit(X, y).cv_results_) > ``` > > Would give > > [image: image.png] > > HTH :) > > > Haha, this is perfect. I didn't know you could pass a list of dicts to > param_grid. > > Thanks! > > > > On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande < > deshpande.jai...@gmail.com> wrote: > > Hi, > > Sometimes when using GridSearchCV, I realize that in the grid there are > certain combinations of hyperparameters that are either incompatible or > redundant. For example, when using an MLP, if I specify the following grid: > > grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant', > 'invscaling', 'adaptive']} > > then it yields the following ParameterGrid: > > [{'learning_rate': 'constant', 'solver': 'sgd'}, > {'learning_rate': 'constant', 'solver': 'adam'}, > {'learning_rate': 'invscaling', 'solver': 'sgd'}, > {'learning_rate': 'invscaling', 'solver': 'adam'}, > {'learning_rate': 'adaptive', 'solver': 'sgd'}, > {'learning_rate': 'adaptive', 'solver': 'adam'}] > > Now, three of these are redundant, since learning_rate is used only for > the sgd solver. Ideally I'd like to specify these cases upfront, and for > that I have a simple hack ( > https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38). > Using that yields a ParameterGrid as follows: > > [{'learning_rate': 'constant', 'solver': 'adam'}, > {'learning_rate': 'invscaling', 'solver': 'adam'}, > {'learning_rate': 'adaptive', 'solver': 'adam'}] > > which is then simply removed from the original ParameterGrid. > > I wonder if there's a simpler way of doing this. Would it help if we had > an additional parameter (something like "grid_exceptions") in GridSearchCV, > which would remove these dicts from the list of parameters? > > Thanks > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > -- > Raghav RV > https://github.com/raghavrv > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn >
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn