Hello Kirill,
> After running this piece of code the predictedResponse matrix was the zero
> matrix (2x2) rather than something close to arma::mat("1 2; 3 4”). What did I
> do
> wrong?
Depending on the network and input the ReLU function might be problematic since
a large gradient could cause the weights to update in such a way that the
network units will never activate. In your case, the Identity function might be
a better solution or the sigmoid function if you are going for logistic
regression. Using the following:
arma::mat data("1 2");
arma::mat trainingResponses("1 2; 3 4");
FFN<MeanSquaredError<>> ffn;
ffn.Add<Linear<>>(1, 2);
ffn.Add<IdentityLayer<> >();
ffn.Train(data, trainingResponses);
arma::mat predictedResponses;
ffn.Predict(data, predictedResponses);
I get the following results:
1.2766 1.8192
2.9841 4.0109
which is close to what you would expect.
> I also have noticed that if I don’t add ReLULayer<>, than there is an error
> during training:
>
> unknown location:0: fatal error: in "CVTest/MSENNTest": signal: SIGABRT
> (application abort requested)
This is a shortcoming that occurs for a single layer network, in this case we
can't store the activation in the upcomming layer.
I hope this is helpful, let me know if I can clarify anything further.
Thanks,
Marcus
> On 29. May 2017, at 15:42, Kirill Mishchenko <[email protected]> wrote:
>
> Hi!
>
> I’m working on cross-validation module for mlpack, and for better code
> coverage in tests I want to check some functionality on neural networks. For
> that I need to train a very simple feedforward neural network that is able to
> remember responses for training data. I tried the following:
>
> arma::mat data("1 2");
> arma::mat trainingResponses("1 2; 3 4");
>
> FFN<MeanSquaredError<>> ffn;
> ffn.Add<Linear<>>(1, 2);
> ffn.Add<ReLULayer<>>();
>
> ffn.Train(data, trainingResponses);
>
> arma::mat predictedResponses;
> ffn.Predict(data, predictedResponses);
>
> After running this piece of code the predictedResponse matrix was the zero
> matrix (2x2) rather than something close to arma::mat("1 2; 3 4”). What did I
> do wrong?
>
> I also have noticed that if I don’t add ReLULayer<>, than there is an error
> during training:
>
> unknown location:0: fatal error: in "CVTest/MSENNTest": signal: SIGABRT
> (application abort requested)
>
> Is it possible to train a linear model with the FNN class (i.e. linear
> regression)?
>
> Best regards,
>
> Kirill Mishchenko
>
>
>
> _______________________________________________
> mlpack mailing list
> [email protected]
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
_______________________________________________
mlpack mailing list
[email protected]
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
