Hello Sebastian, I did normalization of my training set and used the same mean and stdev values to normalize my test set, instead of calculating means and stdev from the test set. I did that because my training set size is finite and the value of each feature is a descriptor that is characteristic of the 3D shape of the observation. The test set would definitely have different mean and stdev values from the training set, and if I had used them to normalize it then I believe I would have distorted the original descriptor values. Anyway, after this normalization I don't get 0 positive predictions anymore by the MLPClassifier.
I still don't understand your second suggestion. I cannot find any parameter to control the epoch or measure the cost in sklearn .neural_network.MLPClassifier. Do you suggest to use your own classes from github instead? Besides that my goal is not to make one MLPClassifier using a specific training set, but rather to write a program that can take as input various training sets each time and and train a neural network that will classify a given test set. Therefore, unless I didn't understand your points, working with 3 arbitrary random_state values on my current training set in order to find one value to yield good predictions, wont solve my problem. best Thomas On 8 December 2016 at 01:19, Sebastian Raschka <[email protected]> wrote: > Hi, Thomas, > we had a related thread on the email list some time ago, let me post it > for reference further below. Regarding your question, I think you may want > make sure that you standardized the features (which makes the learning > generally it less sensitive to learning rate and random weight > initialization). However, even then, I would try at least 1-3 different > random seeds and look at the cost vs time — what can happen is that you > land in different minima depending on the weight initialization as > demonstrated in the example below (in MLPs you have the problem of a > complex cost surface). > > Best, > Sebastian > > The default is set 100 units in the hidden layer, but theoretically, it > should work with 2 hidden logistic units (I think that’s the typical > textbook/class example). I think what happens is that it gets stuck in > local minima depending on the random weight initialization. E.g., the > following works just fine: > > from sklearn.neural_network import MLPClassifier > X = [[0, 0], [0, 1], [1, 0], [1, 1]] > y = [0, 1, 1, 0] > clf = MLPClassifier(solver='lbfgs', > activation='logistic', > alpha=0.0, > hidden_layer_sizes=(2,), > learning_rate_init=0.1, > max_iter=1000, > random_state=20) > clf.fit(X, y) > res = clf.predict([[0, 0], [0, 1], [1, 0], [1, 1]]) > print(res) > print(clf.loss_) > > > but changing the random seed to 1 leads to: > > [0 1 1 1] > 0.34660921283 > > For comparison, I used a more vanilla MLP (1 hidden layer with 2 units and > logistic activation as well; https://github.com/ > rasbt/python-machine-learning-book/blob/master/code/ch12/ch12.ipynb), > essentially resulting in the same problem: > > > > > > > > > > > > > > > > > On Dec 7, 2016, at 6:45 PM, Thomas Evangelidis <[email protected]> wrote: > > I tried the sklearn.neural_network.MLPClassifier with the default > parameters using the input data I quoted in my previous post about > Nu-Support Vector Classifier. The predictions are great but the problem is > that sometimes when I rerun the MLPClassifier it predicts no positive > observations (class 1). I have noticed that this can be controlled by the > random_state parameter, e.g. MLPClassifier(random_state=0) gives always no > positive predictions. My question is how can I chose the right random_state > value in a real blind test case? > > thanks in advance > Thomas > > > -- > ====================================================================== > Thomas Evangelidis > Research Specialist > CEITEC - Central European Institute of Technology > Masaryk University > Kamenice 5/A35/1S081, > 62500 Brno, Czech Republic > > email: [email protected] > [email protected] > > website: https://sites.google.com/site/thomasevangelidishomepage/ > > > _______________________________________________ > scikit-learn mailing list > [email protected] > https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > scikit-learn mailing list > [email protected] > https://mail.python.org/mailman/listinfo/scikit-learn > > -- ====================================================================== Thomas Evangelidis Research Specialist CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/1S081, 62500 Brno, Czech Republic email: [email protected] [email protected] website: https://sites.google.com/site/thomasevangelidishomepage/
_______________________________________________ scikit-learn mailing list [email protected] https://mail.python.org/mailman/listinfo/scikit-learn
