Ok, thank you for your advices. Setting the number of iterations to 0 is probably what I need.
Best regards, Kirill Mishchenko > On 30 May 2017, at 15:00, Marcus Edel <[email protected]> wrote: > > Hello Kirill, > > If you use the MeanSquaredError as performance function you can get > arbitrarily close to the expected mean squared error. But it's not granted you > reach the expected error with the specified architecture or it might take some > time to get there, so you have to experiment with some models that might > converge in an acceptable time. I would expect that: > > FFN<MeanSquaredError<>> ffn; > ffn.Add<IdentityLayer<> >(); > ffn.Add<Linear<>>(1, 8); > ffn.Add<SigmoidLayer<> >(); > ffn.Add<Linear<>>(8, 2); > ffn.Add<IdentityLayer<> >(); > > returns better results in terms of the mean squared error. Anyway, most of the > optimizer classes define a maxIterations parameter that will limit the number > of > iterations. If set to 0 the optimizer does not return before the specified > tolerance is reached. > > I hope this is helpful, if you have any further questions don't hesitate to > ask. > > Thanks, > Marcus > >> On 29. May 2017, at 17:42, Kirill Mishchenko <[email protected] >> <mailto:[email protected]>> wrote: >> >> Thank you, Marcus, it works. >> >> Yet another question: is there any optimizer that let me make responses of >> FFN arbitrarily close to what I expect (arma::mat("1 2; 3 4”))? >> >> Best regards, >> >> Kirill Mishchenko >> >> >>> On 29 May 2017, at 19:30, Marcus Edel <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hello Kirill, >>> >>>> After running this piece of code the predictedResponse matrix was the zero >>>> matrix (2x2) rather than something close to arma::mat("1 2; 3 4”). What >>>> did I do >>>> wrong? >>> >>> Depending on the network and input the ReLU function might be problematic >>> since >>> a large gradient could cause the weights to update in such a way that the >>> network units will never activate. In your case, the Identity function >>> might be >>> a better solution or the sigmoid function if you are going for logistic >>> regression. Using the following: >>> >>> arma::mat data("1 2"); >>> arma::mat trainingResponses("1 2; 3 4"); >>> >>> FFN<MeanSquaredError<>> ffn; >>> ffn.Add<Linear<>>(1, 2); >>> ffn.Add<IdentityLayer<> >(); >>> >>> ffn.Train(data, trainingResponses); >>> >>> arma::mat predictedResponses; >>> ffn.Predict(data, predictedResponses); >>> >>> I get the following results: >>> >>> 1.2766 1.8192 >>> 2.9841 4.0109 >>> >>> which is close to what you would expect. >>> >>>> I also have noticed that if I don’t add ReLULayer<>, than there is an error >>>> during training: >>>> >>>> unknown location:0: fatal error: in "CVTest/MSENNTest": signal: SIGABRT >>>> (application abort requested) >>> >>> This is a shortcoming that occurs for a single layer network, in this case >>> we >>> can't store the activation in the upcomming layer. >>> >>> I hope this is helpful, let me know if I can clarify anything further. >>> >>> Thanks, >>> Marcus >>> >>>> On 29. May 2017, at 15:42, Kirill Mishchenko <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi! >>>> >>>> I’m working on cross-validation module for mlpack, and for better code >>>> coverage in tests I want to check some functionality on neural networks. >>>> For that I need to train a very simple feedforward neural network that is >>>> able to remember responses for training data. I tried the following: >>>> >>>> arma::mat data("1 2"); >>>> arma::mat trainingResponses("1 2; 3 4"); >>>> >>>> FFN<MeanSquaredError<>> ffn; >>>> ffn.Add<Linear<>>(1, 2); >>>> ffn.Add<ReLULayer<>>(); >>>> >>>> ffn.Train(data, trainingResponses); >>>> >>>> arma::mat predictedResponses; >>>> ffn.Predict(data, predictedResponses); >>>> >>>> After running this piece of code the predictedResponse matrix was the zero >>>> matrix (2x2) rather than something close to arma::mat("1 2; 3 4”). What >>>> did I do wrong? >>>> >>>> I also have noticed that if I don’t add ReLULayer<>, than there is an >>>> error during training: >>>> >>>> unknown location:0: fatal error: in "CVTest/MSENNTest": signal: SIGABRT >>>> (application abort requested) >>>> >>>> Is it possible to train a linear model with the FNN class (i.e. linear >>>> regression)? >>>> >>>> Best regards, >>>> >>>> Kirill Mishchenko >>>> >>>> >>>> >>>> _______________________________________________ >>>> mlpack mailing list >>>> [email protected] <mailto:[email protected]> >>>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack >>>> <http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack> >> >
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
