[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-408271045 @marcoabreu The Jenkins CI [1] build is giving an error on : the import statement for MXNet under the Inference Stage of the JenkinsFileForMBCC. I'm not able to figure out why that's happening. Can you have a look at this 69843fbe4d6669c135d3ae85aa56df144bc6c076 and give a second eye opinion ? [1] : http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/test-backwards-compatibility-checker/detail/test-backwards-compatibility-checker/12/pipeline/4/ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-405675518 @marcoabreu ping. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-404623349 @marcoabreu I have removed real world datasets, and instead I am now using randomly generated datasets. I have also removed training part from all the models. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-404271838 @ThomasDelteil What are your thoughts on using real world datasets v/s random ones for this ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-404268363 @marcoabreu I totally agree with you. I'm currently working adding toy examples which use random weights on simple model architectures and run one epoch through them. However, I also wanted to add practical real world models as use cases as well to this. The training time is not an overhead because of small sizes of datasets used and the number of epochs is also small (2 for now). For both of these the goal remains the same : Verify the inference values and assert that they are same. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-404054059 We cannot unittest it as we are not testing the various model save/load APIs on the same version. We are trying to check whether models trained (and saved) on older MXNet versions are still loaded correctly on the latest release candidate of MXNet and thus any new changes that might have crept in remain backwards compatible. @marcoabreu can give more context on this change as well. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-404053135 I am checking the following scenarios to save/load models : 1. Hybridized models import/export API from [Gluon Package (https://mxnet.incubator.apache.org/api/python/gluon/gluon.html) 2. Gluon Models load_params/save_params API from [Gluon Package] (https://mxnet.incubator.apache.org/api/python/gluon/gluon.html) 3. Gluon Models load_parameters/save_parameters API from [Gluon Package] (https://mxnet.incubator.apache.org/api/python/gluon/gluon.html) 4. Declarative Models load_checkpoint() from [Model API] (https://mxnet.incubator.apache.org/api/python/model.html) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker
piyushghai commented on issue #11626: [MXNET-651] MXNet Model Backwards Compatibility Checker URL: https://github.com/apache/incubator-mxnet/pull/11626#issuecomment-404051396 @szha Here's what this PR does : "Backward compatibility aims to check whether models trained on earlier versions of MXNet are loading fine on the latest version or the latest release candidate. It also aims to do a sanity check for consistency in the inference on these trained models. " Here's the relevant JIRA Issue for it : https://issues.apache.org/jira/browse/MXNET-651 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
