Hello Kirill, If you use the MeanSquaredError as performance function you can get arbitrarily close to the expected mean squared error. But it's not granted you reach the expected error with the specified architecture or it might take some time to get there, so you have to experiment with some models that might converge in an acceptable time. I would expect that:
FFN<MeanSquaredError<>> ffn; ffn.Add<IdentityLayer<> >(); ffn.Add<Linear<>>(1, 8); ffn.Add<SigmoidLayer<> >(); ffn.Add<Linear<>>(8, 2); ffn.Add<IdentityLayer<> >(); returns better results in terms of the mean squared error. Anyway, most of the optimizer classes define a maxIterations parameter that will limit the number of iterations. If set to 0 the optimizer does not return before the specified tolerance is reached. I hope this is helpful, if you have any further questions don't hesitate to ask. Thanks, Marcus > On 29. May 2017, at 17:42, Kirill Mishchenko <[email protected]> wrote: > > Thank you, Marcus, it works. > > Yet another question: is there any optimizer that let me make responses of > FFN arbitrarily close to what I expect (arma::mat("1 2; 3 4”))? > > Best regards, > > Kirill Mishchenko > > >> On 29 May 2017, at 19:30, Marcus Edel <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hello Kirill, >> >>> After running this piece of code the predictedResponse matrix was the zero >>> matrix (2x2) rather than something close to arma::mat("1 2; 3 4”). What did >>> I do >>> wrong? >> >> Depending on the network and input the ReLU function might be problematic >> since >> a large gradient could cause the weights to update in such a way that the >> network units will never activate. In your case, the Identity function might >> be >> a better solution or the sigmoid function if you are going for logistic >> regression. Using the following: >> >> arma::mat data("1 2"); >> arma::mat trainingResponses("1 2; 3 4"); >> >> FFN<MeanSquaredError<>> ffn; >> ffn.Add<Linear<>>(1, 2); >> ffn.Add<IdentityLayer<> >(); >> >> ffn.Train(data, trainingResponses); >> >> arma::mat predictedResponses; >> ffn.Predict(data, predictedResponses); >> >> I get the following results: >> >> 1.2766 1.8192 >> 2.9841 4.0109 >> >> which is close to what you would expect. >> >>> I also have noticed that if I don’t add ReLULayer<>, than there is an error >>> during training: >>> >>> unknown location:0: fatal error: in "CVTest/MSENNTest": signal: SIGABRT >>> (application abort requested) >> >> This is a shortcoming that occurs for a single layer network, in this case we >> can't store the activation in the upcomming layer. >> >> I hope this is helpful, let me know if I can clarify anything further. >> >> Thanks, >> Marcus >> >>> On 29. May 2017, at 15:42, Kirill Mishchenko <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi! >>> >>> I’m working on cross-validation module for mlpack, and for better code >>> coverage in tests I want to check some functionality on neural networks. >>> For that I need to train a very simple feedforward neural network that is >>> able to remember responses for training data. I tried the following: >>> >>> arma::mat data("1 2"); >>> arma::mat trainingResponses("1 2; 3 4"); >>> >>> FFN<MeanSquaredError<>> ffn; >>> ffn.Add<Linear<>>(1, 2); >>> ffn.Add<ReLULayer<>>(); >>> >>> ffn.Train(data, trainingResponses); >>> >>> arma::mat predictedResponses; >>> ffn.Predict(data, predictedResponses); >>> >>> After running this piece of code the predictedResponse matrix was the zero >>> matrix (2x2) rather than something close to arma::mat("1 2; 3 4”). What did >>> I do wrong? >>> >>> I also have noticed that if I don’t add ReLULayer<>, than there is an error >>> during training: >>> >>> unknown location:0: fatal error: in "CVTest/MSENNTest": signal: SIGABRT >>> (application abort requested) >>> >>> Is it possible to train a linear model with the FNN class (i.e. linear >>> regression)? >>> >>> Best regards, >>> >>> Kirill Mishchenko >>> >>> >>> >>> _______________________________________________ >>> mlpack mailing list >>> [email protected] <mailto:[email protected]> >>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack >>> <http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack> >
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
