On Wed, 23 Nov 2016 at 17:31 Joel Nothman <joel.noth...@gmail.com> wrote:
> Raghav's example of
>
>
> [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
> 'sgd'}, {'solver': 'adam'}]
>
> was not correct.
>
> Should be
>
>
> [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
> ['sgd']}, {'solver': ['adam']}]
>
> (Note all values of dicts are lists)
>
Ah, thanks!
(Just ran into an error as it started iterating over the "sgd".)
>
> On 23 November 2016 at 22:52, Jaidev Deshpande <deshpande.jai...@gmail.com
> > wrote:
>
>
>
> On Wed, 23 Nov 2016 at 16:29 Raghav R V <rag...@gmail.com> wrote:
>
> Hi!
>
> What you could do is specify lists of dicts to group the parameters which
> apply together in one dict...
>
> [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
> 'sgd'}, {'solver': 'adam'}]
>
> ```py
> from sklearn.neural_network import MLPClassifier
> from sklearn.model_selection import GridSearchCV
> from sklearn.datasets import make_classification
>
> from pandas import DataFrame
>
> X, y = make_classification(random_state=42)
>
> gs = GridSearchCV(MLPClassifier(random_state=42),
> param_grid=[{'learning_rate': ['constant', 'invscaling',
> 'adaptive'],
> 'solver': ['sgd',]},
> {'solver': ['adam',]}])
>
> DataFrame(gs.fit(X, y).cv_results_)
> ```
>
> Would give
>
> [image: image.png]
>
> HTH :)
>
>
> Haha, this is perfect. I didn't know you could pass a list of dicts to
> param_grid.
>
> Thanks!
>
>
>
> On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande <
> deshpande.jai...@gmail.com> wrote:
>
> Hi,
>
> Sometimes when using GridSearchCV, I realize that in the grid there are
> certain combinations of hyperparameters that are either incompatible or
> redundant. For example, when using an MLP, if I specify the following grid:
>
> grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant',
> 'invscaling', 'adaptive']}
>
> then it yields the following ParameterGrid:
>
> [{'learning_rate': 'constant', 'solver': 'sgd'},
> {'learning_rate': 'constant', 'solver': 'adam'},
> {'learning_rate': 'invscaling', 'solver': 'sgd'},
> {'learning_rate': 'invscaling', 'solver': 'adam'},
> {'learning_rate': 'adaptive', 'solver': 'sgd'},
> {'learning_rate': 'adaptive', 'solver': 'adam'}]
>
> Now, three of these are redundant, since learning_rate is used only for
> the sgd solver. Ideally I'd like to specify these cases upfront, and for
> that I have a simple hack (
> https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38).
> Using that yields a ParameterGrid as follows:
>
> [{'learning_rate': 'constant', 'solver': 'adam'},
> {'learning_rate': 'invscaling', 'solver': 'adam'},
> {'learning_rate': 'adaptive', 'solver': 'adam'}]
>
> which is then simply removed from the original ParameterGrid.
>
> I wonder if there's a simpler way of doing this. Would it help if we had
> an additional parameter (something like "grid_exceptions") in GridSearchCV,
> which would remove these dicts from the list of parameters?
>
> Thanks
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn@python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
>
>
> --
> Raghav RV
> https://github.com/raghavrv
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn@python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn@python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn@python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
