Justobe opened a new issue #17686: Inconsistency between Tensorflow, MXNet and Theano backends in LSTM layer URL: https://github.com/apache/incubator-mxnet/issues/17686 ## Description Hi，I find the inconsistency between MXNet,Tensorflow and Theano backends in LSTM layer when I use a simple lstm model for a regression task. I loaded the same model on different backends(tensorflow, mxnet and theano), and the prediction accuracy of mxnet is different from that of tensorflow and theano under the same input. We calculate the MSE between the output of the model and the ground truth. MSE can be calculated as:  The MSE values of tensorflow and theano are 0.00015594294981565327 while the value of mxnet is 0.5000014050006867. I put input data into the model and recorded the ouputs of each layers to see how this happens. It seems that the outputs on tensorflow and theano are very close. But for MXNet, the outputs of the first lstm layer and all the layers after this layer are very different from those in tensorflow and theano. The results are attached below TensorFlow vs Theano  TensorFlow vs MXNet  Theano vs MXNet  The delta column shows the discrepancy, which is calculated as:  Based on the results, the deviation seems to be caused by LSTM of MXNet backend. ## To Reproduce We attach the .h5 file, input and the code below. [code.zip](https://github.com/apache/incubator-mxnet/files/4250250/code.zip) 1. to see the MSE value: ``` python get_model_accurary.py [backend name] ``` 2. to see the output of intermediate layers: ``` python get_layers_output.py [backend name] ``` 3. bug localization: ```shell python get_layers_deviation.py ``` ## What have you tried to solve it? I encountered this problem on mxnet-cu101(Version 1.5.1.post0). When I upgraded mxnet to 1.6, the bug still exists. ## Environment - MXNet: mxnet-cu101==1.5.1.post0 - keras-mxnet: 2.2.4.2 - tensorflow-gpu:1.14.0 - theano:1.0.4 - CUDA: 10.1 You can use the following command to configure the environment ```shell pip install keras-mxnet pip install mxnet-cu101 pip install tensorflow-gpu==1.14.0 pip install theano==1.0.4 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
