[GitHub] larroy commented on a change in pull request #11055: [MXNET-57] Added support android64

[GitBox]

larroy commented on a change in pull request #11055: [MXNET-57] Added support 
android64 
URL: https://github.com/apache/incubator-mxnet/pull/11055#discussion_r194963500
 
 

 ##
 File path: ci/docker/runtime_functions.sh
 ##
 @@ -163,6 +163,30 @@ build_android_arm64() {
 cp dist/*.whl /work/build
 }
 
+build_android_arm64() {
+set -ex
+cd /work/build
+#-DCMAKE_ANDROID_NDK=${CROSS_ROOT} \
+#-DCMAKE_SYSTEM_VERSION=${ANDROID_NDK_REVISION} \
+#-DCMAKE_SYSTEM_NAME=Android \
+cmake\
+-DANDROID=ON \
+-DUSE_CUDA=OFF\
+-DUSE_SSE=OFF\
+-DUSE_LAPACK=OFF\
+-DUSE_OPENCV=OFF\
+-DUSE_OPENMP=OFF\
+-DUSE_SIGNAL_HANDLER=ON\
+-DCMAKE_BUILD_TYPE=RelWithDebInfo\
 
 Review comment:
   I don't see how the points you are making about an experimental build which 
is not official nor tested are of any relevance at this stage of Android 
support. This is a development build and I want debug symbols to diagnose 
crashes, this should be reason enough to have it as it is, because size of 
build doesn't really matter right now, and debugability is more important.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] liuzx32 commented on issue #11221: How to store a large distributed training model with KVStore？

[GitBox]

liuzx32 commented on issue #11221: How to store a large distributed training 
model with KVStore？
URL: 
https://github.com/apache/incubator-mxnet/issues/11221#issuecomment-396822641
 
 
   @marcoabreu Does mxnet support the training model distributed store in every 
worker or server？


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jacob081 commented on issue #11100: Installation instructions for Windows don't work

[GitBox]

jacob081 commented on issue #11100: Installation instructions for Windows don't 
work
URL: 
https://github.com/apache/incubator-mxnet/issues/11100#issuecomment-396820726
 
 
   I also cant not find setupenv.cmd.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] adaaaaaa commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

adaa commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-396820345
 
 
   CUDA is too big to download and install... i'm not going to install and 
remove CUDA now... 
   maybe your computer performance is much better and faster than mine,can you 
try it?
   @szha 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] adaaaaaa commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

adaa commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-396820345
 
 
   CUDA is too big to download and install... i'm not going to install and 
remove CUDA now... 
   
   
   @szha 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] adaaaaaa commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

adaa commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-396818982
 
 
   @chinakook don't know how it compile...my operating system is Manjaro X64 is 
based on Arch...can i figure it out from command line?
   and CUDA is download from software source...both cu91 an cu92


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11245: [MXNET-244][MXNET-523][ARM] improvements to ARMv7 based builds.

[GitBox]

marcoabreu commented on a change in pull request #11245: 
[MXNET-244][MXNET-523][ARM] improvements to ARMv7 based builds.
URL: https://github.com/apache/incubator-mxnet/pull/11245#discussion_r194950646
 
 

 ##
 File path: tests/python/unittest/test_ndarray.py
 ##
 @@ -711,9 +711,8 @@ def get_values(ensure_unique):
  k=dat_size*dat_size*dat_size*dat_size, is_ascend=False)
 assert_almost_equal(nd_ret_argsort, gt)
 
-# test topk with a big shape
-a = mx.nd.arange(0, 54686454, step=1, repeat=1)
-assert_almost_equal(a.topk(k=54686454).asnumpy(), a.asnumpy()[::-1])
+a = mx.nd.arange(0, 1024, step=1, repeat=1)
+assert_almost_equal(a.topk(k=1024).asnumpy(), a.asnumpy()[::-1])
 
 Review comment:
   But your patch effectively defeats the purpose of this test. As you said, it 
was lenghtly discussed and you were always told to write a cpp test and to not 
disable / change this test. So either write a cpp test and remove this one or 
leave it alone.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] larroy commented on a change in pull request #11245: [MXNET-244][MXNET-523][ARM] improvements to ARMv7 based builds.

[GitBox]

larroy commented on a change in pull request #11245: 
[MXNET-244][MXNET-523][ARM] improvements to ARMv7 based builds.
URL: https://github.com/apache/incubator-mxnet/pull/11245#discussion_r194947843
 
 

 ##
 File path: tests/python/unittest/test_ndarray.py
 ##
 @@ -711,9 +711,8 @@ def get_values(ensure_unique):
  k=dat_size*dat_size*dat_size*dat_size, is_ascend=False)
 assert_almost_equal(nd_ret_argsort, gt)
 
-# test topk with a big shape
-a = mx.nd.arange(0, 54686454, step=1, repeat=1)
-assert_almost_equal(a.topk(k=54686454).asnumpy(), a.asnumpy()[::-1])
+a = mx.nd.arange(0, 1024, step=1, repeat=1)
+assert_almost_equal(a.topk(k=1024).asnumpy(), a.asnumpy()[::-1])
 
 Review comment:
   I have discussed at length that is not ok to reserve a 4gb ndarray. The 
author that introduced this test should have done a cpp unit test for the 
specific regression that he targetted. This is blocking raspberry pi 
validation. We have customers that need to run the core unit tests on device, 
as well as we do. Please don't block our work of bringing mxnet to resource 
constraint devices.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] uran209 commented on issue #5432: Predicting with mx.io.ImageRecordIter object returns different size vector each time is run

[GitBox]

uran209 commented on issue #5432: Predicting with mx.io.ImageRecordIter object 
returns different size vector each time is run
URL: 
https://github.com/apache/incubator-mxnet/issues/5432#issuecomment-396798033
 
 
   I have the same problem of "Predicting with mx.io.ImageRecordIter object 
returns different size vector".
   Could you give me some helps if you had solved this? Thanks in advance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Erdos001 opened a new issue #11255: P100 + MXNet is slow

[GitBox]

Erdos001 opened a new issue #11255: P100 + MXNet is slow
URL: https://github.com/apache/incubator-mxnet/issues/11255
 
 
   Note: Providing complete information in the most concise form is the best 
way to get help. This issue template serves as the checklist for essential 
information to most of the technical issues and bug reports. For non-technical 
issues and feature requests, feel free to present the information in what you 
believe is the best form.
   
   For Q & A and discussion, please start a discussion thread at 
https://discuss.mxnet.io 
   
   ## Description
   I have installed the Anaconda3 and installed the mxnet-cu90 by pip but the 
training speed is very slow
   
   and I find that ,the data reading speed is much slow , when reading image 
data and decoding the binary data by imdecode , the GPU usage is 30%, I guess 
that, the speed bottleneck of the speed is data reading.
   
   multi-gpu and single gpu are the same slow 
   ## Environment info (Required)
   Ubuntu16.04
   P100
   
   
   ```
   What to do:
   1. Download the diagnosis script from 
https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py
   2. Run the script using `python diagnose.py` and paste its output here.
   
   ```
   
   Package used (Python/R/Scala/Julia):
   (I'm using ...)
   
   For Scala user, please provide:
   1. Java version: (`java -version`)
   2. Maven version: (`mvn -version`)
   3. Scala runtime if applicable: (`scala -version`)
   
   For R user, please provide R `sessionInfo()`:
   
   ## Build info (Required if built from source)
   
   Compiler (gcc/clang/mingw/visual studio):
   
   MXNet commit hash:
   (Paste the output of `git rev-parse HEAD` here.)
   
   Build config:
   (Paste the content of config.mk, or the build command.)
   pip install mxnet-cu90
   ## Error Message:
   (Paste the complete error message, including stack trace.)
   No error, but the speed is slow
   
   
![default](https://user-images.githubusercontent.com/12196464/41326818-77fd52e6-6ef3-11e8-9a3e-7b8c146306e8.png)
   
   ## Minimum reproducible example
   (If you are using your own code, please provide a short script that 
reproduces the error. Otherwise, please provide link to the existing example.)
   
   ## Steps to reproduce
   (Paste the commands you ran that produced the error.)
   
   1.Training with mxnet and P100
   2.
   
   ## What have you tried to solve it?
   
   1. installed python3, python2
   2.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pengzhao-intel commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox]

pengzhao-intel commented on issue #11129: [MXNET-497]Test kAddTo request for 
mkldnn operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#issuecomment-396792242
 
 
   @azai91, You can try to figure out the root cause and fix it with a new PR. 
I don't think we need to revert the previous PR.  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] leezu commented on issue #11223: Allow specifying AdaGrad initial accumulator value

[GitBox]

leezu commented on issue #11223: Allow specifying AdaGrad initial accumulator 
value
URL: https://github.com/apache/incubator-mxnet/pull/11223#issuecomment-396791591
 
 
   Tests added


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sxjscience commented on issue #11254: add blocklist

[GitBox]

sxjscience commented on issue #11254: add blocklist
URL: https://github.com/apache/incubator-mxnet/pull/11254#issuecomment-396791367
 
 
   Should solve https://github.com/apache/incubator-mxnet/issues/10101, 
https://github.com/apache/incubator-mxnet/issues/9264


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] azai91 commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox]

azai91 commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn 
operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#issuecomment-396790290
 
 
   @pengzhao-intel is anyone working on a PR to revert the regression or should 
I work on?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] azai91 commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox]

azai91 commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn 
operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#issuecomment-396790290
 
 
   @pengzhao-intel is anyone working on a PR to revert the regression or is 
someone already reverting?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #11254: add blocklist

[GitBox]

szha commented on a change in pull request #11254: add blocklist
URL: https://github.com/apache/incubator-mxnet/pull/11254#discussion_r194935377
 
 

 ##
 File path: docs/api/python/gluon/gluon.md
 ##
 @@ -116,6 +116,8 @@ net.hybridize()
 
 Block
 HybridBlock
+BlockList
+HybridBlockList
 SymbolBlock
 nn.Sequential
 nn.HybridSequential
 
 Review comment:
   Should these two inherit BlockLists?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong opened a new pull request #11254: add blocklist

[GitBox]

piiswrong opened a new pull request #11254: add blocklist
URL: https://github.com/apache/incubator-mxnet/pull/11254
 
 
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] azai91 commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox]

azai91 commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn 
operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#issuecomment-396784674
 
 
   okay. will add MKLDNN benchmark tests in a separate PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yunzhe-tao opened a new issue #11253: CUDNN implementation of LS

[GitBox]

yunzhe-tao opened a new issue #11253: CUDNN implementation of LS
URL: https://github.com/apache/incubator-mxnet/issues/11253
 
 
   Hi, I want to implement the LSTM projection layer on MXNet Gluon. I was 
trying to implement the LSTMP layer similar to gluon.rnn.lstm based on 
gluon.contrib.rnn.LSTMPCell but I found that the ndarray.rnn function in 
LSTM._forward_kernel is implemented based on C++ and the codes are not 
available. Moreover, the function only works for RNN, LSTM and GRU. Without 
that function, I found LSTMP layer runs really slow. So I am wondering if 
someone can implement _forward_kernel for more general recurrent layers. Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 opened a new pull request #11252: [WIP] [MXNET-323] Improve performance of broadcast ops backward pass

[GitBox]

anirudh2290 opened a new pull request #11252: [WIP] [MXNET-323] Improve 
performance of broadcast ops backward pass
URL: https://github.com/apache/incubator-mxnet/pull/11252
 
 
   ## Description ##
   
   This PR tries to improve the performance of broadcast ops backward pass, by 
caching the intermediate computations and using LaunchEx. The speedup for both 
forward and backward pass combined for the broadcast_add is around 1.4X. The 
experiments have been run on p2.8xlarge.
   
   The below numbers are for broadcasting a tensor of shape (1,) to a tensor of 
shape destination shape as given below.
   
   | Destination tensor shape | Before the Change | After the Change |
   | - | --- | - |
   | 100, 100 | 0.017 | 0.013 |
   | 250, 250 | 0.085 | 0.065 |
   | 500, 500 | 0.35 | 0.25 |
   | 1000, 1000 | 1.34 | 0.98 |
   | 2000, 2000 | 5.6 | 3.9 |
   | 12000, 12000 | 188 | 144 |
   | 2**17, 10, 10 | 18 | 12 |
   
   ```
   import numpy as np
   import mxnet as mx
   import time
   
   a = mx.sym.var('a')
   b = mx.sym.var('b')
   
   a_ = mx.nd.ones((2**17, 10, 10))
   b_ = mx.nd.ones((1))
   
   func2 = mx.sym.broadcast_add(a, b).bind(mx.cpu(), args={'a': a_, 'b': b_}, 
args_grad = {'a': mx.nd.ones((2**17, 10, 10)), 'b': mx.nd.ones((1))})
   
   for _ in range(2):
   # boadcast_add(array, array)
   start = time.time()
   for i in range(100):
   out = func2.forward(is_train=True)[0]
   func2.backward(mx.nd.ones((2**17, 10, 10)))
   mx.nd.waitall()
   print("func2: {}".format(time.time() - start))
   
   ```
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] reminisce opened a new pull request #11251: [WIP] Subgraph op

[GitBox]

reminisce opened a new pull request #11251: [WIP] Subgraph op
URL: https://github.com/apache/incubator-mxnet/pull/11251
 
 
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   
   @piiswrong @zheng-da 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pengzhao-intel commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox]

pengzhao-intel commented on issue #11129: [MXNET-497]Test kAddTo request for 
mkldnn operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#issuecomment-396779411
 
 
   @azai91 Really thanks your efforts for improving the quality of MKL-DNN 
backend.
   
   @azai91 @zheng-da @eric-haibin-lin @marcoabreu
   
   Two suggestions:
   
   **1) avoiding change the MKL-DNN implementations when adding the test 
cases.** 
  It will be hard to track back the changes. If it really needs to change 
(for the bugfix or other reasons), I think we can file another PR.
   
  One example is #11026, this PR is marked as adding test cases, so we 
don't pay much attention because it's always a good thing for more cases. But 
the implementation was changed and led to a performance regression for 
resnet-50 inference (from 238 drop to 179), see below.
   
   **2) running the `benchmark_score.py` to verify the performance for each of 
CI**
   
   ```
   commit 92286c9106dd63d2bfd062f9abb0e53b071a46e4
   Author: Alexander Zai aza...@gmail.com
   Date:   Tue May 29 17:48:33 2018 -0700
   
   [lvtao@mlt-skx052 image-classification]$ python benchmark_score.py
   INFO:root:network: resnet-50
   INFO:root:device: cpu(0)
   /home/lvtao/Workspace/mxnet-official/python/mxnet/module/base_module.py:66: 
UserWarning: Data provided by label_shapes don't match names specified by 
label_names ([] vs. ['softmax_label'])
 warnings.warn(msg)
   [22:09:59] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 9437184 bytes 
with malloc directly
   INFO:root:batch size  1, image/sec: 83.039246
   INFO:root:batch size  2, image/sec: 118.022742
   INFO:root:batch size  4, image/sec: 149.483457
   INFO:root:batch size  8, image/sec: 162.208255
   INFO:root:batch size 16, image/sec: 166.800091
   INFO:root:batch size 32, image/sec: 165.821744
   INFO:root:batch size 64, image/sec: 175.934763
   INFO:root:batch size 128, image/sec: 179.160898
   INFO:root:batch size 256, image/sec: 178.767242
   
   
   commit 9514a1e39f8356f8fee6202cd86c8f20fbf301b6
   Author: kpmurali 37911926+kpmur...@users.noreply.github.com
   Date:   Tue May 29 17:36:35 2018 -0700
   
   [lvtao@mlt-skx052 image-classification]$ python benchmark_score.py
   INFO:root:network: resnet-50
   INFO:root:device: cpu(0)
   /home/lvtao/Workspace/mxnet-official/python/mxnet/module/base_module.py:66: 
UserWarning: Data provided by label_shapes don't match names specified by 
label_names ([] vs. ['softmax_label'])
 warnings.warn(msg)
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 147456 bytes 
with malloc directly
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 589824 bytes 
with malloc directly
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 2359296 bytes 
with malloc directly
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 9437184 bytes 
with malloc directly
   INFO:root:batch size  1, image/sec: 89.205967
   INFO:root:batch size  2, image/sec: 128.775569
   INFO:root:batch size  4, image/sec: 156.363125
   INFO:root:batch size  8, image/sec: 195.993911
   INFO:root:batch size 16, image/sec: 219.215664
   INFO:root:batch size 32, image/sec: 224.414152
   INFO:root:batch size 64, image/sec: 238.657344
   INFO:root:batch size 128, image/sec: 230.266770
   INFO:root:batch size 256, image/sec: 225.139635
   
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #8219: Broadcasting ops are slow

[GitBox]

anirudh2290 commented on issue #8219: Broadcasting ops are slow
URL: 
https://github.com/apache/incubator-mxnet/issues/8219#issuecomment-395926419
 
 
   I have been looking at this issue. MXNet forward pass for broadcast_add is 
much faster than tensorflow and pytorch. To give some numbers here (Experiments 
conducted on my setup of p2.8xlarge, Testing for CPU performance):
   
   For broadcasting a tensor of shape (1,) to  a tensor of shape (2**17, 10, 
10), only forward pass:
   pytorch: 0.6 seconds
   mxnet: 0.4 seconds
   tensorflow: 2.1 seconds.
   
   When we include both forward and backward pass: 
   pytorch:  5.1 seconds
   mxnet: 16 seconds
   tensorflow: 2.2 seconds
   
   So we decide to look at the MXNet backward pass and try out some 
optimizations. We try out using LaunchEx so that each thread gets a bigger 
chunk of workload. This by itself doesn't help. The bottleneck for the backward 
pass is the for loop that runs for each thread: 
https://github.com/apache/incubator-mxnet/blob/master/src/operator/tensor/broadcast_reduce-inl.h#L164
   
   There are a lot of repeated computations for computing coords and dot inside 
the for loop. We try to cache this computation. This improves the speed more 
than 1.4X to around 12 seconds for mxnet. This involves extra memory which is 
the drawback. You can see the rough implementation here: 
https://github.com/anirudh2290/mxnet/blob/cached_broadcast/src/operator/tensor/broadcast_reduce-inl.h#L233
   
   We suspect (not yet confirmed) that Tensorflow uses the eigen library to do 
the reduce. This is something that we should run experiments for and consider 
as a replacement as we deprecate mshadow.
   
   Next steps:
   1. Introduce the cached solution to MXNet as a stop gap solution.
   2. Investigate eigen library and whether it will be worthwhile to add it as 
a MXNet dependency.
   
   Here is the script. Please let me know if there is any caveat that I have 
missed:
   
   ```
   import numpy as np
   import mxnet as mx
   
   a = mx.sym.var('a')
   b = mx.sym.var('b')
   
   a_ = mx.nd.ones((2**17, 10, 10))
   b_ = mx.nd.ones((1,))
   
   func2 = mx.sym.broadcast_add(a, b).bind(mx.cpu(), args={'a': a_, 'b': b_}, 
args_grad = {'a': mx.nd.ones((2**17, 10, 10)), 'b': mx.nd.ones((1))})
   
   for _ in range(4):
   # boadcast_add(array, array)
   start = time.time()
   for i in range(100):
   out = func2.forward(is_train=True)[0]
   func2.backward(mx.nd.ones((2**17, 10, 10)))
   mx.nd.waitall()
   print("mxnet time taken is: {}".format(time.time() - start))
   
   import torch
   
   for i in range(4):
   
   start = time.time()
   for j in range(100):
   x = torch.ones((2**17, 10, 10), requires_grad=True)
   y = torch.ones((1), requires_grad=True)
   z = x + y
   z.backward(torch.ones((2**17, 10, 10)), retain_graph=True)
   print("torch time taken is {}".format(time.time() - start))
   
   import tensorflow as tf
   a = tf.ones([2**17, 10, 10], name='a')
   b = tf.ones([1], name='b')
   add_op = a + b
   g = tf.gradients(add_op, [a,b])
   
   for x in range(4):
   with tf.Session() as session:
   start = time.time()
   for i in range(1, 100):
   grad_vals = session.run(g)
   print("tf time taken is: {}".format(time.time() - start))
   
   ```
   
   @piiswrong @andreaolgiati @srochel 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ctcyang commented on issue #11237: Bring back MXNET_GPU_COPY_NTHREADS env variable

[GitBox]

ctcyang commented on issue #11237: Bring back MXNET_GPU_COPY_NTHREADS env 
variable
URL: https://github.com/apache/incubator-mxnet/pull/11237#issuecomment-396778091
 
 
   When it's set to 1, copies between GPUs are serialized. Even between 2 GPUs, 
if GPU 0 is currently sending something to 1 on one stream, and GPU 1 wants to 
send something to 0, the latter send will have to wait. Assuming bidirectional 
NVLink, the 0 -> 1 link can only go up to 48 GB/s, so the 48 GB/s in reverse 
direction is unused.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ctcyang commented on issue #11237: Bring back MXNET_GPU_COPY_NTHREADS env variable

[GitBox]

ctcyang commented on issue #11237: Bring back MXNET_GPU_COPY_NTHREADS env 
variable
URL: https://github.com/apache/incubator-mxnet/pull/11237#issuecomment-396778091
 
 
   When it's set to 1, copies between GPUs are serialized. Even between 2 GPUs, 
if GPU 0 is currently sending something to 1 on one stream, and GPU 1 wants to 
send something to 0, the latter send will have to wait leaving the 1 -> 0 link 
under-utilized.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 closed pull request #11123: [MXNET-319][DO NOT MERGE][Deprecated] Javadoc fix

[GitBox]

lanking520 closed pull request #11123: [MXNET-319][DO NOT MERGE][Deprecated] 
Javadoc fix
URL: https://github.com/apache/incubator-mxnet/pull/11123
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/scala-package/core/pom.xml b/scala-package/core/pom.xml
index 361bfab5d61..724ffbb369d 100644
--- a/scala-package/core/pom.xml
+++ b/scala-package/core/pom.xml
@@ -65,7 +65,24 @@
 org.scalastyle
 scalastyle-maven-plugin
   
+  
+net.alchim31.maven
+scala-maven-plugin
+3.3.2
+
+
+
+  
+package
+attach-javadocs
+
+  doc-jar
+
+  
+
+  
 
+
   
   
 
diff --git 
a/scala-package/core/src/main/scala/org/apache/mxnet/EvalMetric.scala 
b/scala-package/core/src/main/scala/org/apache/mxnet/EvalMetric.scala
index de2881a221c..4556d4f7bfa 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/EvalMetric.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/EvalMetric.scala
@@ -108,9 +108,9 @@ class Accuracy extends EvalMetric("accuracy") {
 
 for ((pred, label) <- preds zip labels) {
   val predLabel = if (pred.shape == label.shape) {
-NDArray.argmax(Map("axis" -> 1, "keepdims" -> true))(pred)
+NDArray.api.argmax(pred, Some(1), Some(true))
   } else {
-NDArray.argmax_channel(pred)
+NDArray.api.argmax_channel(pred)
   }
   require(label.shape == predLabel.shape,
 s"label ${label.shape} and prediction ${predLabel.shape}" +
@@ -172,7 +172,7 @@ class F1 extends EvalMetric("f1") {
   "labels and predictions should have the same length.")
 
 for ((pred, label) <- preds zip labels) {
-  val predLabel = NDArray.argmax_channel(pred)
+  val predLabel = NDArray.api.argmax_channel(pred)
   require(label.shape == predLabel.shape,
 s"label ${label.shape} and prediction ${predLabel.shape}" +
 s"should have the same length.")
@@ -232,7 +232,7 @@ class Perplexity(ignoreLabel: Option[Int] = None, axis: Int 
= -1) extends EvalMe
   require(label.size == pred.size / pred.shape.toArray.reverse.head,
 s"shape mismatch: ${label.shape} vs. ${pred.shape}")
   val l = 
label.asInContext(pred.context).asType(DType.Int32).reshape(Shape(label.size))
-  val p = NDArray.pick(Map("axis" -> this.axis))(pred, label)
+  val p = NDArray.api.pick(pred, label, Some(this.axis))
   probs += p.head
 }
 
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Monitor.scala 
b/scala-package/core/src/main/scala/org/apache/mxnet/Monitor.scala
index 8e53d652fde..25ae374aa39 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Monitor.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Monitor.scala
@@ -38,7 +38,7 @@ class Monitor(
 
   if (statFunc == null) {
 statFunc = (x: NDArray) => {
-  NDArray.norm(x) / math.sqrt(x.size.toDouble).toFloat
+  NDArray.api.norm(x) / math.sqrt(x.size.toDouble).toFloat
 }
   }
 
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala 
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 469107aa58c..49f4d35136f 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -65,12 +65,12 @@ object NDArray {
 val ndArgs = ArrayBuffer.empty[NDArray]
 val posArgs = ArrayBuffer.empty[String]
 args.foreach {
-  case arr: NDArray =>
-ndArgs.append(arr)
-  case arrFunRet: NDArrayFuncReturn =>
-arrFunRet.arr.foreach(ndArgs.append(_))
-  case arg =>
-posArgs.append(arg.toString)
+case arr: NDArray =>
+  ndArgs.append(arr)
+case arrFunRet: NDArrayFuncReturn =>
+  arrFunRet.arr.foreach(ndArgs.append(_))
+case arg =>
+  posArgs.append(arg.toString)
 }
 
 require(posArgs.length <= function.arguments.length,
@@ -81,6 +81,7 @@ object NDArray {
 ++ function.arguments.slice(0, posArgs.length).zip(posArgs) - "out"
   ).map { case (k, v) => k -> v.toString }
 
+
 val (oriOutputs, outputVars) =
   if (kwargs != null && kwargs.contains("out")) {
 val output = kwargs("out")
@@ -537,6 +538,10 @@ object NDArray {
 new NDArray(handleRef.value)
   }
 
+  private def _crop_assign(kwargs: Map[String, Any] = null)(args: Any*) : 
NDArrayFuncReturn = {
+genericNDArrayFunctionInvoke("_crop_assign", args, kwargs)
+  }
+
   // TODO: imdecode
 }
 
diff --git 
a/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
 

[GitHub] lanking520 commented on issue #11123: [MXNET-319][DO NOT MERGE][Deprecated] Javadoc fix

[GitBox]

lanking520 commented on issue #11123: [MXNET-319][DO NOT MERGE][Deprecated] 
Javadoc fix
URL: https://github.com/apache/incubator-mxnet/pull/11123#issuecomment-396778128
 
 
   Master branch merged, no need to go with this solution


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11198: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

lanking520 commented on issue #11198: [MXNET-531][Work In Progress] MNIST 
Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198#issuecomment-396777833
 
 
   Abandon this PR since it includes too much conflict with Master Branch, open 
one here: https://github.com/apache/incubator-mxnet/pull/11250


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 closed pull request #11198: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

lanking520 closed pull request #11198: [MXNET-531][Work In Progress] MNIST 
Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/3rdparty/ps-lite b/3rdparty/ps-lite
index 8a763892a97..a6dda54604a 16
--- a/3rdparty/ps-lite
+++ b/3rdparty/ps-lite
@@ -1 +1 @@
-Subproject commit 8a763892a973afc1acd3d4b469d05bb338a83a6e
+Subproject commit a6dda54604a07d1fb21b016ed1e3f4246b08222a
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala 
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 49f4d35136f..f55ae3f0d08 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -72,7 +72,6 @@ object NDArray {
 case arg =>
   posArgs.append(arg.toString)
 }
-
 require(posArgs.length <= function.arguments.length,
   s"len(posArgs) = ${posArgs.length}, should be less or equal to 
len(arguments) " +
   s"= ${function.arguments.length}")
diff --git 
a/scala-package/examples/src/main/scala/org/apache/mxnet/examples/imclassification/ModelTrain.scala
 
b/scala-package/examples/src/main/scala/org/apache/mxnet/examples/imclassification/ModelTrain.scala
new file mode 100644
index 000..1fa3ecaefaf
--- /dev/null
+++ 
b/scala-package/examples/src/main/scala/org/apache/mxnet/examples/imclassification/ModelTrain.scala
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.examples.imclassification
+
+import org.apache.mxnet.Callback.Speedometer
+import org.apache.mxnet._
+import org.apache.mxnet.optimizer.SGD
+import org.slf4j.LoggerFactory
+
+object ModelTrain {
+  private val logger = LoggerFactory.getLogger(classOf[ModelTrain])
+
+  // scalastyle:off parameterNum
+  def fit(dataDir: String, batchSize: Int, numExamples: Int, devs: 
Array[Context],
+  network: Symbol, dataLoader: (String, Int, KVStore) => (DataIter, 
DataIter),
+  kvStore: String, numEpochs: Int, modelPrefix: String = null, 
loadEpoch: Int = -1,
+  lr: Float = 0.1f, lrFactor: Float = 1f, lrFactorEpoch: Float = 1f,
+  clipGradient: Float = 0f, monitorSize: Int = -1): Accuracy = {
+// kvstore
+var kv = KVStore.create(kvStore)
+
+// load model
+val modelPrefixWithRank =
+  if (modelPrefix == null) null
+  else modelPrefix + s"-${kv.rank}"
+
+val (argParams, auxParams, beginEpoch) =
+  if (loadEpoch >= 0) {
+require(modelPrefixWithRank != null)
+val tmp = FeedForward.load(modelPrefix, loadEpoch)
+(tmp.getArgParams, tmp.getAuxParams, loadEpoch)
+  } else {
+(null, null, 0)
+  }
+
+// save model
+val checkpoint: EpochEndCallback =
+  if (modelPrefix == null) null
+  else new EpochEndCallback {
+override def invoke(epoch: Int, symbol: Symbol,
+argParams: Map[String, NDArray],
+auxStates: Map[String, NDArray]): Unit = {
+  Model.saveCheckpoint(modelPrefix, epoch + 1, symbol, argParams, 
auxParams)
+}
+  }
+
+// data
+val (train, validation) = dataLoader(dataDir, batchSize, kv)
+
+// train
+val epochSize =
+  if (kvStore == "dist_sync") numExamples / batchSize / kv.numWorkers
+  else numExamples / batchSize
+
+val lrScheduler =
+  if (lrFactor < 1f) {
+new FactorScheduler(step = Math.max((epochSize * lrFactorEpoch).toInt, 
1),
+factor = lrFactor)
+  } else {
+null
+  }
+val optimizer: Optimizer = new SGD(learningRate = lr,
+lrScheduler = lrScheduler, clipGradient = clipGradient,
+momentum = 0.9f, wd = 0.1f)
+
+// disable kvstore for single device
+if (kv.`type`.contains("local") && (devs.length == 1 || devs(0).deviceType 
!= "gpu")) {
+  kv.dispose()
+  kv = null
+}

[GitHub] lanking520 opened a new pull request #11250: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

lanking520 opened a new pull request #11250: [MXNET-531][Work In Progress] 
MNIST Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11250
 
 
   ## Description ##
   This PR contains examples written from the new API.
   @nswamy @yzhliu @andrewfayres
   All examples pushed here are tested locally on my Mac. It lives in the new 
package called `org.apache.mxnet.examples`
   
   ## Checklist ##
   ### Essentials ###
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: [MXNET-386] ongoing maintenance on NDArray (#11126)

[liuyizhi]

This is an automated email from the ASF dual-hosted git repository.

liuyizhi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 74479b8  [MXNET-386] ongoing maintenance on NDArray (#11126)
74479b8 is described below

commit 74479b89eaba8241573079aa5e32f0ba0f8dd00e
Author: Lanking 
AuthorDate: Tue Jun 12 17:06:13 2018 -0700

[MXNET-386] ongoing maintenance on NDArray (#11126)

* Important ndarray feature

* merge generic function Invoke

* Pass the Scala Style test

* add Experimental tags

* Change with NDArgs addition

* change dir for Experimental tag

* reTrigger CI

* add Symbol Macros change

* Add some workaround on NDArray

* Simplify the base part

* add changes on ND and Symbols...

* avoid vars

* add Symbol Macros

* Trigger the CI

* Trigger CI
---
 .../src/main/scala/org/apache/mxnet/NDArray.scala  | 17 +---
 .../org/apache/mxnet/annotation/Experimental.scala | 25 
 .../scala/org/apache/mxnet/APIDocGenerator.scala   |  7 +++-
 .../main/scala/org/apache/mxnet/NDArrayMacro.scala | 47 +-
 .../main/scala/org/apache/mxnet/SymbolMacro.scala  | 24 ---
 5 files changed, 87 insertions(+), 33 deletions(-)

diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala 
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 469107a..49f4d35 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -65,12 +65,12 @@ object NDArray {
 val ndArgs = ArrayBuffer.empty[NDArray]
 val posArgs = ArrayBuffer.empty[String]
 args.foreach {
-  case arr: NDArray =>
-ndArgs.append(arr)
-  case arrFunRet: NDArrayFuncReturn =>
-arrFunRet.arr.foreach(ndArgs.append(_))
-  case arg =>
-posArgs.append(arg.toString)
+case arr: NDArray =>
+  ndArgs.append(arr)
+case arrFunRet: NDArrayFuncReturn =>
+  arrFunRet.arr.foreach(ndArgs.append(_))
+case arg =>
+  posArgs.append(arg.toString)
 }
 
 require(posArgs.length <= function.arguments.length,
@@ -81,6 +81,7 @@ object NDArray {
 ++ function.arguments.slice(0, posArgs.length).zip(posArgs) - "out"
   ).map { case (k, v) => k -> v.toString }
 
+
 val (oriOutputs, outputVars) =
   if (kwargs != null && kwargs.contains("out")) {
 val output = kwargs("out")
@@ -537,6 +538,10 @@ object NDArray {
 new NDArray(handleRef.value)
   }
 
+  private def _crop_assign(kwargs: Map[String, Any] = null)(args: Any*) : 
NDArrayFuncReturn = {
+genericNDArrayFunctionInvoke("_crop_assign", args, kwargs)
+  }
+
   // TODO: imdecode
 }
 
diff --git 
a/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
 
b/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
new file mode 100644
index 000..33d1d33
--- /dev/null
+++ 
b/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.annotation
+
+import java.lang.annotation.{ElementType, Retention, Target, _}
+
+@Retention(RetentionPolicy.RUNTIME)
+@Target(Array(ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, 
ElementType.PARAMETER,
+  ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE))
+class Experimental {}
diff --git 
a/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala 
b/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
index 90fe260..3bbc7fd 100644
--- a/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
+++ b/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
@@ -52,8 +52,9 @@ private[mxnet] object APIDocGenerator{
 val apacheLicence = "/*\n* Licensed to the Apache Software Foundation 
(ASF) under one or more\n* contributor license 

[GitHub] yzhliu closed pull request #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

yzhliu closed pull request #11126: [MXNET-386] ongoing maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala 
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 469107aa58c..49f4d35136f 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -65,12 +65,12 @@ object NDArray {
 val ndArgs = ArrayBuffer.empty[NDArray]
 val posArgs = ArrayBuffer.empty[String]
 args.foreach {
-  case arr: NDArray =>
-ndArgs.append(arr)
-  case arrFunRet: NDArrayFuncReturn =>
-arrFunRet.arr.foreach(ndArgs.append(_))
-  case arg =>
-posArgs.append(arg.toString)
+case arr: NDArray =>
+  ndArgs.append(arr)
+case arrFunRet: NDArrayFuncReturn =>
+  arrFunRet.arr.foreach(ndArgs.append(_))
+case arg =>
+  posArgs.append(arg.toString)
 }
 
 require(posArgs.length <= function.arguments.length,
@@ -81,6 +81,7 @@ object NDArray {
 ++ function.arguments.slice(0, posArgs.length).zip(posArgs) - "out"
   ).map { case (k, v) => k -> v.toString }
 
+
 val (oriOutputs, outputVars) =
   if (kwargs != null && kwargs.contains("out")) {
 val output = kwargs("out")
@@ -537,6 +538,10 @@ object NDArray {
 new NDArray(handleRef.value)
   }
 
+  private def _crop_assign(kwargs: Map[String, Any] = null)(args: Any*) : 
NDArrayFuncReturn = {
+genericNDArrayFunctionInvoke("_crop_assign", args, kwargs)
+  }
+
   // TODO: imdecode
 }
 
diff --git 
a/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
 
b/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
new file mode 100644
index 000..33d1d330979
--- /dev/null
+++ 
b/scala-package/core/src/main/scala/org/apache/mxnet/annotation/Experimental.scala
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.annotation
+
+import java.lang.annotation.{ElementType, Retention, Target, _}
+
+@Retention(RetentionPolicy.RUNTIME)
+@Target(Array(ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, 
ElementType.PARAMETER,
+  ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE))
+class Experimental {}
diff --git 
a/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala 
b/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
index 90fe2604e8b..3bbc7fd6a90 100644
--- a/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
+++ b/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
@@ -52,8 +52,9 @@ private[mxnet] object APIDocGenerator{
 val apacheLicence = "/*\n* Licensed to the Apache Software Foundation 
(ASF) under one or more\n* contributor license agreements.  See the NOTICE file 
distributed with\n* this work for additional information regarding copyright 
ownership.\n* The ASF licenses this file to You under the Apache License, 
Version 2.0\n* (the \"License\"); you may not use this file except in 
compliance with\n* the License.  You may obtain a copy of the License at\n*\n*  
  http://www.apache.org/licenses/LICENSE-2.0\n*\n* Unless required by 
applicable law or agreed to in writing, software\n* distributed under the 
License is distributed on an \"AS IS\" BASIS,\n* WITHOUT WARRANTIES OR 
CONDITIONS OF ANY KIND, either express or implied.\n* See the License for the 
specific language governing permissions and\n* limitations under the 
License.\n*/\n"
 val scalaStyle = "// scalastyle:off"
 val packageDef = "package org.apache.mxnet"
+val imports = "import org.apache.mxnet.annotation.Experimental"
 val absClassDef = s"abstract class $packageName"
-val finalStr = 

[GitHub] yzhliu commented on issue #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

yzhliu commented on issue #11126: [MXNET-386] ongoing maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126#issuecomment-396772387
 
 
   @nswamy as we talked offline, we can later figure out the way to enable 
compile-time log for Experimental. I do the merge so that other work rely on 
this PR can continue. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on a change in pull request #11198: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

lanking520 commented on a change in pull request #11198: [MXNET-531][Work In 
Progress] MNIST Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198#discussion_r194921916
 
 

 ##
 File path: 
scala-package/examples/src/test/scala/org/apache/mxnet/examples/imclassification/MNISTExampleSuite.scala
 ##
 @@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.examples.imclassification
+
+import java.io.File
+
+import org.apache.mxnet.Context
+import org.scalatest.{BeforeAndAfterAll, FunSuite}
+import org.slf4j.LoggerFactory
+
+import scala.sys.process.Process
+
+/**
+  * Integration test for imageClassifier example.
+  * This will run as a part of "make scalatest"
+  */
+class MNISTExampleSuite extends FunSuite with BeforeAndAfterAll {
+  private val logger = LoggerFactory.getLogger(classOf[MNISTExampleSuite])
+
+  test("Example CI: Test MNIST Training") {
+// This test is CPU only
+if (System.getenv().containsKey("SCALA_TEST_ON_GPU") &&
+  System.getenv("SCALA_TEST_ON_GPU").toInt == 1) {
+  logger.info("CPU test only, skipped...")
+} else {
+  logger.info("Downloading mnist model")
+  val tempDirPath = System.getProperty("java.io.tmpdir")
+  val modelDirPath = tempDirPath + File.separator + "mnist/"
+  logger.info("tempDirPath: %s".format(tempDirPath))
+  Process("wget 
https://s3.us-east-2.amazonaws.com/mxnet-scala/scala-example-ci; +
 
 Review comment:
   Good idea, Will try change into this: 
https://stackoverflow.com/questions/24162478/how-to-download-and-save-a-file-from-the-internet-using-scala


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11198: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

marcoabreu commented on a change in pull request #11198: [MXNET-531][Work In 
Progress] MNIST Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198#discussion_r194921289
 
 

 ##
 File path: 
scala-package/examples/src/test/scala/org/apache/mxnet/examples/imclassification/MNISTExampleSuite.scala
 ##
 @@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.examples.imclassification
+
+import java.io.File
+
+import org.apache.mxnet.Context
+import org.scalatest.{BeforeAndAfterAll, FunSuite}
+import org.slf4j.LoggerFactory
+
+import scala.sys.process.Process
+
+/**
+  * Integration test for imageClassifier example.
+  * This will run as a part of "make scalatest"
+  */
+class MNISTExampleSuite extends FunSuite with BeforeAndAfterAll {
+  private val logger = LoggerFactory.getLogger(classOf[MNISTExampleSuite])
+
+  test("Example CI: Test MNIST Training") {
+// This test is CPU only
+if (System.getenv().containsKey("SCALA_TEST_ON_GPU") &&
+  System.getenv("SCALA_TEST_ON_GPU").toInt == 1) {
+  logger.info("CPU test only, skipped...")
+} else {
+  logger.info("Downloading mnist model")
+  val tempDirPath = System.getProperty("java.io.tmpdir")
+  val modelDirPath = tempDirPath + File.separator + "mnist/"
+  logger.info("tempDirPath: %s".format(tempDirPath))
+  Process("wget 
https://s3.us-east-2.amazonaws.com/mxnet-scala/scala-example-ci; +
 
 Review comment:
   This will break on windows (I know we don't support it yet) and requires 
wget. Are there any scala-tools that allow downloading files with features like 
retry-mechanisms etc?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11198: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

marcoabreu commented on a change in pull request #11198: [MXNET-531][Work In 
Progress] MNIST Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198#discussion_r194921345
 
 

 ##
 File path: 
scala-package/examples/src/test/scala/org/apache/mxnet/examples/imclassification/MNISTExampleSuite.scala
 ##
 @@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.examples.imclassification
+
+import java.io.File
+
+import org.apache.mxnet.Context
+import org.scalatest.{BeforeAndAfterAll, FunSuite}
+import org.slf4j.LoggerFactory
+
+import scala.sys.process.Process
+
+/**
+  * Integration test for imageClassifier example.
+  * This will run as a part of "make scalatest"
+  */
+class MNISTExampleSuite extends FunSuite with BeforeAndAfterAll {
+  private val logger = LoggerFactory.getLogger(classOf[MNISTExampleSuite])
+
+  test("Example CI: Test MNIST Training") {
+// This test is CPU only
+if (System.getenv().containsKey("SCALA_TEST_ON_GPU") &&
+  System.getenv("SCALA_TEST_ON_GPU").toInt == 1) {
+  logger.info("CPU test only, skipped...")
+} else {
+  logger.info("Downloading mnist model")
+  val tempDirPath = System.getProperty("java.io.tmpdir")
+  val modelDirPath = tempDirPath + File.separator + "mnist/"
+  logger.info("tempDirPath: %s".format(tempDirPath))
+  Process("wget 
https://s3.us-east-2.amazonaws.com/mxnet-scala/scala-example-ci; +
 
 Review comment:
   We should try to avoid using the Process-Method and rather rely on 
language-specific tools


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11199: [MXNET-538] Add XUnit to python tests

[GitBox]

marcoabreu commented on a change in pull request #11199: [MXNET-538] Add XUnit 
to python tests
URL: https://github.com/apache/incubator-mxnet/pull/11199#discussion_r194920366
 
 

 ##
 File path: Jenkinsfile
 ##
 @@ -465,46 +479,69 @@ try {
 parallel 'Python2: CPU': {
   node('mxnetlinux-cpu') {
 ws('workspace/ut-python2-cpu') {
-  init_git()
-  unpack_lib('cpu')
-  python2_ut('ubuntu_cpu')
+  try {
+init_git()
+unpack_lib('cpu')
+python2_ut('ubuntu_cpu')
+  } finally {
+collect_test_results_unix('nosetests1.xml', 
'nosetests_python2_cpu1.xml')
+collect_test_results_unix('nosetests2.xml', 
'nosetests_python2_cpu2.xml')
+collect_test_results_unix('nosetests3.xml', 
'nosetests_python2_cpu3.xml')
+  }
 }
   }
 },
 'Python3: CPU': {
   node('mxnetlinux-cpu') {
 ws('workspace/ut-python3-cpu') {
-  init_git()
-  unpack_lib('cpu')
-  python3_ut('ubuntu_cpu')
+  try {
+init_git()
+unpack_lib('cpu')
+python3_ut('ubuntu_cpu')
+  } finally {
+collect_test_results_unix('nosetests1.xml', 
'nosetests_python3_cpu1.xml')
 
 Review comment:
   They will be available in the web UI like here: 
http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/view/change-requests/job/PR-11199/8/
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11199: [MXNET-538] Add XUnit to python tests

[GitBox]

marcoabreu commented on a change in pull request #11199: [MXNET-538] Add XUnit 
to python tests
URL: https://github.com/apache/incubator-mxnet/pull/11199#discussion_r194920455
 
 

 ##
 File path: ci/docker/runtime_functions.sh
 ##
 @@ -500,14 +502,14 @@ unittest_ubuntu_gpu_R() {
 unittest_centos7_cpu() {
 set -ex
 cd /work/mxnet
-python3.6 -m "nose" --with-timer --verbose tests/python/unittest
-python3.6 -m "nose" --with-timer --verbose tests/python/train
+python3.6 -m "nose" --with-xunit --xunit-file nosetests1.xml --verbose 
tests/python/unittest
+python3.6 -m "nose" --with-xunit --xunit-file nosetests2.xml --verbose 
tests/python/train
 }
 
 unittest_centos7_gpu() {
 set -ex
 cd /work/mxnet
-python3.6 -m "nose" --with-timer --verbose tests/python/gpu
 
 Review comment:
   It only creates the summary at the end of the tests and bloats up the error 
messages (if there are any), but it's not really of use. The timing will be 
tracked anyways


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #11199: [MXNET-538] Add XUnit to python tests

[GitBox]

eric-haibin-lin commented on a change in pull request #11199: [MXNET-538] Add 
XUnit to python tests
URL: https://github.com/apache/incubator-mxnet/pull/11199#discussion_r194919887
 
 

 ##
 File path: Jenkinsfile
 ##
 @@ -465,46 +479,69 @@ try {
 parallel 'Python2: CPU': {
   node('mxnetlinux-cpu') {
 ws('workspace/ut-python2-cpu') {
-  init_git()
-  unpack_lib('cpu')
-  python2_ut('ubuntu_cpu')
+  try {
+init_git()
+unpack_lib('cpu')
+python2_ut('ubuntu_cpu')
+  } finally {
+collect_test_results_unix('nosetests1.xml', 
'nosetests_python2_cpu1.xml')
+collect_test_results_unix('nosetests2.xml', 
'nosetests_python2_cpu2.xml')
+collect_test_results_unix('nosetests3.xml', 
'nosetests_python2_cpu3.xml')
+  }
 }
   }
 },
 'Python3: CPU': {
   node('mxnetlinux-cpu') {
 ws('workspace/ut-python3-cpu') {
-  init_git()
-  unpack_lib('cpu')
-  python3_ut('ubuntu_cpu')
+  try {
+init_git()
+unpack_lib('cpu')
+python3_ut('ubuntu_cpu')
+  } finally {
+collect_test_results_unix('nosetests1.xml', 
'nosetests_python3_cpu1.xml')
 
 Review comment:
   are these xml files accessible from web UI? how are they going to be 
presented?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #11199: [MXNET-538] Add XUnit to python tests

[GitBox]

eric-haibin-lin commented on a change in pull request #11199: [MXNET-538] Add 
XUnit to python tests
URL: https://github.com/apache/incubator-mxnet/pull/11199#discussion_r194920022
 
 

 ##
 File path: ci/docker/runtime_functions.sh
 ##
 @@ -500,14 +502,14 @@ unittest_ubuntu_gpu_R() {
 unittest_centos7_cpu() {
 set -ex
 cd /work/mxnet
-python3.6 -m "nose" --with-timer --verbose tests/python/unittest
-python3.6 -m "nose" --with-timer --verbose tests/python/train
+python3.6 -m "nose" --with-xunit --xunit-file nosetests1.xml --verbose 
tests/python/unittest
+python3.6 -m "nose" --with-xunit --xunit-file nosetests2.xml --verbose 
tests/python/train
 }
 
 unittest_centos7_gpu() {
 set -ex
 cd /work/mxnet
-python3.6 -m "nose" --with-timer --verbose tests/python/gpu
 
 Review comment:
   why remove `--with-timer`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu closed issue #11221: How to store a large distributed training model with KVStore？

[GitBox]

marcoabreu closed issue #11221: How to store a large distributed training model 
with KVStore？
URL: https://github.com/apache/incubator-mxnet/issues/11221
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194919544
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
+ "prefix-symbol.json, prefix-epoch.params")
+raise
+
+sym, arg_params, aux_params = mx.model.load_checkpoint(model_name, 
num_epochs)
+trained_model = mx.mod.Module(symbol=sym, label_names=None)
+trained_model.bind(data_shapes=[('data', input_shape[0])],
+   label_shapes=None, for_training=False)
+
+# Merging arg and aux parameters
+params = {}
+params.update(arg_params)
+params.update(aux_params)
+
 
 Review comment:
   removed


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11237: Bring back MXNET_GPU_COPY_NTHREADS env variable

[GitBox]

marcoabreu commented on issue #11237: Bring back MXNET_GPU_COPY_NTHREADS env 
variable
URL: https://github.com/apache/incubator-mxnet/pull/11237#issuecomment-396767385
 
 
   Would it make sense to choose this value dependent on the number of GPUs 
available?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11247: add seed_aug parameter for ImageRecordItr to fix random seed for augm…

[GitBox]

marcoabreu commented on issue #11247: add seed_aug parameter for ImageRecordItr 
to fix random seed for augm…
URL: https://github.com/apache/incubator-mxnet/pull/11247#issuecomment-396767163
 
 
   Hi @wenyangchu ,
   
   thanks a lot for your contribution! Would you mind adding a test to ensure 
that the behaviour is actually deterministic now and that everything is 
behaving as expected? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194918505
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_onnx.py
 ##
 @@ -0,0 +1,270 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# 
https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/mx2onnx_converter.py#
+#  Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+#  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+#  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+#  PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+#  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+#  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+#  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+#  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=invalid-name,too-many-locals,no-self-use,too-many-arguments,
+# pylint: disable=maybe-no-member,too-many-nested-blocks
+"""MXNet to ONNX graph converter functions"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import json
+import numpy as np
+
+from  import context
+from  import ndarray as nd
+from  import io
+from  import module as mod
+
+
+class MXNetGraph(object):
+"""Class to convert MXNet to ONNX graph"""
+registry_ = {}
+input_output_maps_ = {}
+
+def __init__(self):
+# topologically sorted nodes
+self.nodes = []
+self.input_tensors = []
+self.output_tensors = []
+
+@staticmethod
+def register(op_name):
+"""Register operator"""
+def wrapper(func):
+"""Helper function to map functions"""
+MXNetGraph.registry_[op_name] = func
+return func
+
+return wrapper
+
+@staticmethod
+def convert_layer(node, **kwargs):
+"""Convert MXNet layer to ONNX"""
+op = str(node["op"])
+if op not in MXNetGraph.registry_:
+raise AttributeError("No conversion function registered for op 
type %s yet." % op)
+convert_fun = MXNetGraph.registry_[op]
+return convert_fun(node, **kwargs)
+
+@staticmethod
+def forward_pass(inputs, sym, arg_params, aux_params):
+""" Do a forward pass based on the sym and params"""
+data_names = [graph_input for graph_input in sym.list_inputs()
+  if graph_input not in arg_params and graph_input not in 
aux_params
+  and graph_input != 'softmax_label']
+
+data_shapes = []
+# Adding extra dimension of batch_size 1 if the batch_size is 
different for multiple inputs.
+for idx, input_name in enumerate(data_names):
+data_shapes.append((input_name, inputs[idx].shape))
+
+# create module, passing cpu context
+ctx = context.cpu()
+test_mod = mod.Module(symbol=sym, data_names=data_names, context=ctx, 
label_names=None)
+

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194918283
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_onnx.py
 ##
 @@ -0,0 +1,270 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# 
https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/mx2onnx_converter.py#
+#  Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+#  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+#  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+#  PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+#  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+#  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+#  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+#  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=invalid-name,too-many-locals,no-self-use,too-many-arguments,
+# pylint: disable=maybe-no-member,too-many-nested-blocks
+"""MXNet to ONNX graph converter functions"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import json
+import numpy as np
+
+from  import context
+from  import ndarray as nd
+from  import io
+from  import module as mod
+
+
+class MXNetGraph(object):
+"""Class to convert MXNet to ONNX graph"""
+registry_ = {}
+input_output_maps_ = {}
+
+def __init__(self):
+# topologically sorted nodes
+self.nodes = []
+self.input_tensors = []
+self.output_tensors = []
+
+@staticmethod
+def register(op_name):
+"""Register operator"""
+def wrapper(func):
+"""Helper function to map functions"""
+MXNetGraph.registry_[op_name] = func
+return func
+
+return wrapper
+
+@staticmethod
+def convert_layer(node, **kwargs):
+"""Convert MXNet layer to ONNX"""
+op = str(node["op"])
+if op not in MXNetGraph.registry_:
+raise AttributeError("No conversion function registered for op 
type %s yet." % op)
+convert_fun = MXNetGraph.registry_[op]
+return convert_fun(node, **kwargs)
+
+@staticmethod
+def forward_pass(inputs, sym, arg_params, aux_params):
+""" Do a forward pass based on the sym and params"""
+data_names = [graph_input for graph_input in sym.list_inputs()
+  if graph_input not in arg_params and graph_input not in 
aux_params
+  and graph_input != 'softmax_label']
 
 Review comment:
   if label is not provided, MXNet adds label "softmax_label" by default while 
load_checkpoint which is not actually a graph input. So ignoring it here


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact 

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194917649
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_model.py
 ##
 @@ -0,0 +1,89 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# coding: utf-8
+#pylint: disable-msg=too-many-arguments
+
+"""export function"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import logging
+import numpy as np
+
+from base import string_types
+from .export_onnx import MXNetGraph
+from .export_helper import load_module
+
+
+def export_model(model, weights, input_shape, input_type=np.float32,
+ onnx_file_path='model.onnx', log=False):
+"""Exports the MXNet model file, passed as a parameter, into ONNX model.
+Accepts both symbol,parameter objects as well as json and params filepaths 
as input.
+Operator support and coverage - 
https://cwiki.apache.org/confluence/display/MXNET/ONNX
+
+Parameters
+--
+model : str or symbol object
+Path to the json file or Symbol object
+weights : str or symbol object
+Path to the params file or Params object. (Including both arg_params 
and aux_params)
+input_shape : List of tuple
+Input shape of the model e.g (1,3,224,224)
+input_type :
+Input data type e.g. np.float32
+onnx_file_path : str
+Path where to save the generated onnx file
+log : Boolean
+If true will print logs of the model conversion
+
+Returns
+---
+onnx_file_path : str
+Onnx file path
+"""
+
+try:
+from onnx import helper, mapping
+except ImportError:
+raise ImportError("Onnx and protobuf need to be installed. "
+  + "Instructions to install - 
https://github.com/onnx/onnx;)
+
+converter = MXNetGraph()
+
+data_format = np.dtype(input_type)
+if isinstance(model, string_types) and isinstance(weights, string_types):
+logging.info("Converting json and params file to sym and weights")
+sym, params = load_module(model, weights, input_shape)
+onnx_graph = converter.create_onnx_graph_proto(sym, params, 
input_shape,
+   
mapping.NP_TYPE_TO_TENSOR_TYPE[data_format],
+   log=log)
+else:
+onnx_graph = converter.create_onnx_graph_proto(model, weights, 
input_shape,
+   
mapping.NP_TYPE_TO_TENSOR_TYPE[data_format],
+   log=log)
+# Create the model (ModelProto)
+onnx_model = helper.make_model(onnx_graph)
+
+# Save model on disk
+with open(onnx_file_path, "wb") as file_handle:
+serialized = onnx_model.SerializeToString()
+file_handle.write(serialized)
+logging.info("Exported ONNX file %s saved to disk", onnx_file_path)
 
 Review comment:
   Added


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194917602
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_model.py
 ##
 @@ -0,0 +1,89 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# coding: utf-8
+#pylint: disable-msg=too-many-arguments
+
+"""export function"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import logging
+import numpy as np
+
+from base import string_types
+from .export_onnx import MXNetGraph
+from .export_helper import load_module
+
+
+def export_model(model, weights, input_shape, input_type=np.float32,
+ onnx_file_path='model.onnx', log=False):
+"""Exports the MXNet model file, passed as a parameter, into ONNX model.
+Accepts both symbol,parameter objects as well as json and params filepaths 
as input.
+Operator support and coverage - 
https://cwiki.apache.org/confluence/display/MXNET/ONNX
+
+Parameters
+--
+model : str or symbol object
+Path to the json file or Symbol object
+weights : str or symbol object
+Path to the params file or Params object. (Including both arg_params 
and aux_params)
+input_shape : List of tuple
+Input shape of the model e.g (1,3,224,224)
+input_type :
+Input data type e.g. np.float32
+onnx_file_path : str
+Path where to save the generated onnx file
+log : Boolean
+If true will print logs of the model conversion
+
+Returns
+---
+onnx_file_path : str
+Onnx file path
+"""
+
+try:
+from onnx import helper, mapping
+except ImportError:
+raise ImportError("Onnx and protobuf need to be installed. "
+  + "Instructions to install - 
https://github.com/onnx/onnx;)
+
+converter = MXNetGraph()
+
+data_format = np.dtype(input_type)
+if isinstance(model, string_types) and isinstance(weights, string_types):
+logging.info("Converting json and params file to sym and weights")
+sym, params = load_module(model, weights, input_shape)
+onnx_graph = converter.create_onnx_graph_proto(sym, params, 
input_shape,
+   
mapping.NP_TYPE_TO_TENSOR_TYPE[data_format],
+   log=log)
+else:
 
 Review comment:
   Raising error for corner case scenario 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #11212: cherry-pick bug fixes in MKLDNN for v1.2.0

[GitBox]

anirudh2290 commented on issue #11212: cherry-pick bug fixes in MKLDNN for 
v1.2.0
URL: https://github.com/apache/incubator-mxnet/pull/11212#issuecomment-396765184
 
 
   @zheng-da ok got it!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194916325
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_model.py
 ##
 @@ -0,0 +1,89 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# coding: utf-8
+#pylint: disable-msg=too-many-arguments
+
+"""export function"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import logging
+import numpy as np
+
+from base import string_types
+from .export_onnx import MXNetGraph
+from .export_helper import load_module
+
+
+def export_model(model, weights, input_shape, input_type=np.float32,
+ onnx_file_path='model.onnx', log=False):
+"""Exports the MXNet model file, passed as a parameter, into ONNX model.
+Accepts both symbol,parameter objects as well as json and params filepaths 
as input.
+Operator support and coverage - 
https://cwiki.apache.org/confluence/display/MXNET/ONNX
+
+Parameters
+--
+model : str or symbol object
+Path to the json file or Symbol object
+weights : str or symbol object
+Path to the params file or Params object. (Including both arg_params 
and aux_params)
+input_shape : List of tuple
+Input shape of the model e.g (1,3,224,224)
+input_type :
+Input data type e.g. np.float32
+onnx_file_path : str
+Path where to save the generated onnx file
+log : Boolean
+If true will print logs of the model conversion
+
+Returns
+---
+onnx_file_path : str
+Onnx file path
+"""
+
+try:
+from onnx import helper, mapping
+except ImportError:
+raise ImportError("Onnx and protobuf need to be installed. "
+  + "Instructions to install - 
https://github.com/onnx/onnx;)
+
+converter = MXNetGraph()
+
+data_format = np.dtype(input_type)
+if isinstance(model, string_types) and isinstance(weights, string_types):
+logging.info("Converting json and params file to sym and weights")
+sym, params = load_module(model, weights, input_shape)
+onnx_graph = converter.create_onnx_graph_proto(sym, params, 
input_shape,
+   
mapping.NP_TYPE_TO_TENSOR_TYPE[data_format],
+   log=log)
+else:
 
 Review comment:
   We can add this as a future API.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194916115
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_onnx.py
 ##
 @@ -0,0 +1,270 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# 
https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/mx2onnx_converter.py#
+#  Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+#  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+#  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+#  PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+#  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+#  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+#  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+#  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=invalid-name,too-many-locals,no-self-use,too-many-arguments,
+# pylint: disable=maybe-no-member,too-many-nested-blocks
+"""MXNet to ONNX graph converter functions"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import json
+import numpy as np
+
+from  import context
+from  import ndarray as nd
+from  import io
+from  import module as mod
+
+
+class MXNetGraph(object):
+"""Class to convert MXNet to ONNX graph"""
+registry_ = {}
+input_output_maps_ = {}
+
+def __init__(self):
+# topologically sorted nodes
+self.nodes = []
+self.input_tensors = []
+self.output_tensors = []
+
+@staticmethod
+def register(op_name):
+"""Register operator"""
+def wrapper(func):
+"""Helper function to map functions"""
+MXNetGraph.registry_[op_name] = func
+return func
+
+return wrapper
+
+@staticmethod
+def convert_layer(node, **kwargs):
+"""Convert MXNet layer to ONNX"""
+op = str(node["op"])
+if op not in MXNetGraph.registry_:
+raise AttributeError("No conversion function registered for op 
type %s yet." % op)
+convert_fun = MXNetGraph.registry_[op]
+return convert_fun(node, **kwargs)
+
+@staticmethod
+def forward_pass(inputs, sym, arg_params, aux_params):
+""" Do a forward pass based on the sym and params"""
+data_names = [graph_input for graph_input in sym.list_inputs()
+  if graph_input not in arg_params and graph_input not in 
aux_params
+  and graph_input != 'softmax_label']
+
+data_shapes = []
+# Adding extra dimension of batch_size 1 if the batch_size is 
different for multiple inputs.
+for idx, input_name in enumerate(data_names):
+data_shapes.append((input_name, inputs[idx].shape))
+
+# create module, passing cpu context
+ctx = context.cpu()
+test_mod = mod.Module(symbol=sym, data_names=data_names, context=ctx, 
label_names=None)
+

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194915770
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
+ "prefix-symbol.json, prefix-epoch.params")
+raise
+
+sym, arg_params, aux_params = mx.model.load_checkpoint(model_name, 
num_epochs)
+trained_model = mx.mod.Module(symbol=sym, label_names=None)
+trained_model.bind(data_shapes=[('data', input_shape[0])],
 
 Review comment:
   Code removed


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on issue #11212: cherry-pick bug fixes in MKLDNN for v1.2.0

[GitBox]

zheng-da commented on issue #11212: cherry-pick bug fixes in MKLDNN for v1.2.0
URL: https://github.com/apache/incubator-mxnet/pull/11212#issuecomment-396762289
 
 
   this test was from the PR
   https://github.com/apache/incubator-mxnet/pull/10578/files.
   
   
   On Tue, Jun 12, 2018 at 4:01 PM, Anirudh Subramanian <
   notificati...@github.com> wrote:
   
   > *@anirudh2290* commented on this pull request.
   > --
   >
   > In tests/cpp/operator/mkldnn.cc
   > 

   > :
   >
   > > @@ -81,13 +81,6 @@ TEST(MKLDNN_UTIL_FUNC, AlignMem) {
   >  #endif
   >  }
   >
   > -TEST(MKLDNN_UTIL_FUNC, MemFormat) {
   > -  // Check whether the number of format is correct.
   > -  CHECK_EQ(mkldnn_format_last, 56);
   > -  CHECK_EQ(mkldnn_nchw, 5);
   > -  CHECK_EQ(mkldnn_oihw, 12);
   >
   > why are these unnecessary tests and how are they running on master ?
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194914843
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194914781
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
 
 Review comment:
   Keeping epochs to 0 if not provided with the model name


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on a change in pull request #11212: cherry-pick bug fixes in MKLDNN for v1.2.0

[GitBox]

anirudh2290 commented on a change in pull request #11212: cherry-pick bug fixes 
in MKLDNN for v1.2.0
URL: https://github.com/apache/incubator-mxnet/pull/11212#discussion_r194913920
 
 

 ##
 File path: tests/cpp/operator/mkldnn.cc
 ##
 @@ -81,13 +81,6 @@ TEST(MKLDNN_UTIL_FUNC, AlignMem) {
 #endif
 }
 
-TEST(MKLDNN_UTIL_FUNC, MemFormat) {
-  // Check whether the number of format is correct.
-  CHECK_EQ(mkldnn_format_last, 56);
-  CHECK_EQ(mkldnn_nchw, 5);
-  CHECK_EQ(mkldnn_oihw, 12);
 
 Review comment:
   why are these unnecessary tests and how are they running on master ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #10974: Cpp CI may be broken

[GitBox]

szha closed issue #10974: Cpp CI may be broken
URL: https://github.com/apache/incubator-mxnet/issues/10974
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #10974: Cpp CI may be broken

[GitBox]

szha commented on issue #10974: Cpp CI may be broken
URL: 
https://github.com/apache/incubator-mxnet/issues/10974#issuecomment-396760757
 
 
   This has been fixed by #11065 thanks to @haojin2 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #10973: Flaky test_deconvolution

[GitBox]

szha commented on issue #10973: Flaky test_deconvolution
URL: 
https://github.com/apache/incubator-mxnet/issues/10973#issuecomment-396760496
 
 
   Got a couple of this too 
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11041/82/pipeline/724


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #10406: Flaky test - test_gluon_data.test_datasets

[GitBox]

szha commented on issue #10406: Flaky test - test_gluon_data.test_datasets
URL: 
https://github.com/apache/incubator-mxnet/issues/10406#issuecomment-396760237
 
 
   This test has been stable since #11181 thanks to @ThomasDelteil 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #10406: Flaky test - test_gluon_data.test_datasets

[GitBox]

szha closed issue #10406: Flaky test - test_gluon_data.test_datasets
URL: https://github.com/apache/incubator-mxnet/issues/10406
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #11202: Flaky test Python 3: Win: test_operator.test_gru_bidirectional

[GitBox]

szha closed issue #11202: Flaky test Python 3: Win: 
test_operator.test_gru_bidirectional 
URL: https://github.com/apache/incubator-mxnet/issues/11202
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #11202: Flaky test Python 3: Win: test_operator.test_gru_bidirectional

[GitBox]

szha commented on issue #11202: Flaky test Python 3: Win: 
test_operator.test_gru_bidirectional 
URL: 
https://github.com/apache/incubator-mxnet/issues/11202#issuecomment-396759995
 
 
   duplicate #11219 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194912635
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194912618
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194912472
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194912437
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on issue #11219: Flaky test: test_gru_bidirectional

[GitBox]

ThomasDelteil commented on issue #11219: Flaky test: test_gru_bidirectional
URL: 
https://github.com/apache/incubator-mxnet/issues/11219#issuecomment-396758583
 
 
   
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11225/4/pipeline


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11198: [MXNET-531][Work In Progress] MNIST Examples for Scala new API

[GitBox]

lanking520 commented on issue #11198: [MXNET-531][Work In Progress] MNIST 
Examples for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198#issuecomment-396758571
 
 
   @nswamy Done that, this one MNIST specifically
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194908833
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
+ "prefix-symbol.json, prefix-epoch.params")
+raise
+
+sym, arg_params, aux_params = mx.model.load_checkpoint(model_name, 
num_epochs)
+trained_model = mx.mod.Module(symbol=sym, label_names=None)
 
 Review comment:
   done. extracting from sym and params


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194908752
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
+ "prefix-symbol.json, prefix-epoch.params")
+raise
+
+sym, arg_params, aux_params = mx.model.load_checkpoint(model_name, 
num_epochs)
+trained_model = mx.mod.Module(symbol=sym, label_names=None)
 
 Review comment:
   ONNX models are context agnostic. For running an ONNX model you will import 
ONNX model to ur framework model format i.e sym and params  for MXNet. Now you 
use mxnet api to set your context and either retrain this model(i.e ignore the 
params file ) or use it for inference.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dmidge8 commented on issue #4002: SoftmaxOutput and the tutorial is confusing

[GitBox]

dmidge8 commented on issue #4002: SoftmaxOutput and the tutorial is confusing
URL: 
https://github.com/apache/incubator-mxnet/issues/4002#issuecomment-396750449
 
 
   Yes, I concur. I got trapped in the same pitfall. I would have thought that 
the loss function would have been outside of network definition. And even 
knowing that it is in the network definition, I would have put it outside of a 
layer definition.
   And I would add that the doc won't necessary help, since there are several 
languages supported, thus several docs. And I would not necessary think about 
consulting the documentation of another language that does not have the same 
philosophy (and a distant API). And the doc in C++ is kinda small - I think 
some other basics should be covered before explaining that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dmidge8 commented on issue #4002: SoftmaxOutput and the tutorial is confusing

[GitBox]

dmidge8 commented on issue #4002: SoftmaxOutput and the tutorial is confusing
URL: 
https://github.com/apache/incubator-mxnet/issues/4002#issuecomment-396750449
 
 
   Yes, I concur. I got trapped in the same pitfall. I would have thought that 
the loss function would have been outside of network definition. And even 
knowing that it is in the network definition, I would have put it outside of a 
layer definition.
   And I would add that the doc won't necessary help, since the doc in C++ is 
... small, to say the least.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dmidge8 commented on issue #4002: SoftmaxOutput and the tutorial is confusing

[GitBox]

dmidge8 commented on issue #4002: SoftmaxOutput and the tutorial is confusing
URL: 
https://github.com/apache/incubator-mxnet/issues/4002#issuecomment-396750449
 
 
   Yes, I concur. I got trapped in the same pitfall. I would have thought that 
the loss function would have been outside of network definition. And even 
knowing that it is in the network definition, I would have put it outside of a 
layer definition.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy commented on issue #8387: test_operator_gpu.test_cuda_rtc failure - Jetson TX2 (GPU/CUDA)

[GitBox]

sandeep-krishnamurthy commented on issue #8387: test_operator_gpu.test_cuda_rtc 
failure - Jetson TX2 (GPU/CUDA)
URL: 
https://github.com/apache/incubator-mxnet/issues/8387#issuecomment-396748616
 
 
   Resolving. Please reopen if closed in error.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy closed issue #8387: test_operator_gpu.test_cuda_rtc failure - Jetson TX2 (GPU/CUDA)

[GitBox]

sandeep-krishnamurthy closed issue #8387: test_operator_gpu.test_cuda_rtc 
failure - Jetson TX2 (GPU/CUDA)
URL: https://github.com/apache/incubator-mxnet/issues/8387
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy commented on issue #8143: `mx.callback.Speedometer` cannot be used in `epoch_end_callback`

[GitBox]

sandeep-krishnamurthy commented on issue #8143: `mx.callback.Speedometer` 
cannot be used in `epoch_end_callback`
URL: 
https://github.com/apache/incubator-mxnet/issues/8143#issuecomment-396748104
 
 
   @vikingMei - Please reopen if your issue is still not resolved.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194901801
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/op_translations.py
 ##
 @@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+#  https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+#  Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+#  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+#  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+#  PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+#  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+#  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+#  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+#  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"], 
kwargs["in_shape"])
+return [tval_node]
+
+
+def parse_helper(attrs, attrs_name, alt_value=None):
+"""Helper function to parse operator attributes in required format."""
+tuple_re = re.compile('\([0-9L|,| ]+\)')
+if attrs is None:
+return alt_value
+attrs_str = str(attrs.get(attrs_name))
+if attrs_str is None:
+return alt_value
+attrs_match = tuple_re.search(attrs_str)
+if attrs_match is not None:
+if attrs_match.span() == (0, len(attrs_str)):
+dims = eval(attrs_str)
+return dims
+else:
+raise AttributeError("Malformed %s dimensions: %s" % (attrs_name, 
str(attrs_str)))
+return alt_value
+
+
+@mx_op.register("Convolution")
+def convert_convolution(node, **kwargs):
+"""Map MXNet's convolution operator 

[GitHub] sandeep-krishnamurthy closed issue #8143: `mx.callback.Speedometer` cannot be used in `epoch_end_callback`

[GitBox]

sandeep-krishnamurthy closed issue #8143: `mx.callback.Speedometer` cannot be 
used in `epoch_end_callback`
URL: https://github.com/apache/incubator-mxnet/issues/8143
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy closed issue #8060: Evaluation measure remains after several iterations, epochs...What could be the reason?

[GitBox]

sandeep-krishnamurthy closed issue #8060: Evaluation measure remains after 
several iterations, epochs...What could be the reason?
URL: https://github.com/apache/incubator-mxnet/issues/8060
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy commented on issue #8060: Evaluation measure remains after several iterations, epochs...What could be the reason?

[GitBox]

sandeep-krishnamurthy commented on issue #8060: Evaluation measure remains 
after several iterations, epochs...What could be the reason?
URL: 
https://github.com/apache/incubator-mxnet/issues/8060#issuecomment-396747873
 
 
   Please reopen if you cannot find answer for your questions at 
https://discuss.mxnet.io/


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy commented on issue #7997: Question about benchmarking imagenet

[GitBox]

sandeep-krishnamurthy commented on issue #7997: Question about benchmarking 
imagenet
URL: 
https://github.com/apache/incubator-mxnet/issues/7997#issuecomment-396747320
 
 
   Please reopen if you cannot find the required answer.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy closed issue #7997: Question about benchmarking imagenet

[GitBox]

sandeep-krishnamurthy closed issue #7997: Question about benchmarking imagenet
URL: https://github.com/apache/incubator-mxnet/issues/7997
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on a change in pull request #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox]

zheng-da commented on a change in pull request #11129: [MXNET-497]Test kAddTo 
request for mkldnn operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#discussion_r194900964
 
 

 ##
 File path: tests/cpp/operator/mkldnn.cc
 ##
 @@ -571,9 +613,8 @@ void VerifyActResult(const std::vector 
_arrs, const NDArray )
   mshadow::default_real_t *d1 = 
static_cast(blob1.dptr_);
   mshadow::default_real_t *d2 = 
static_cast(blob2.dptr_);
   EXPECT_EQ(tmp1.shape().Size(), tmp2.shape().Size());
-  for (size_t i = 0; i < tmp1.shape().Size(); i++) {
-EXPECT_EQ(d1[i], std::fmax(d2[i], 0));
-  }
+  for (size_t i = 0; i < tmp1.shape().Size(); i++)
+ASSERT_EQ(d2[i], std::fmax(d1[i], 0));
 
 Review comment:
   ASSERT_EQ is defined by mxnet. We should use EXPECT_EQ in the unit test 
because we want to see all failures.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy closed issue #7860: mx.nd.Custom bug

[GitBox]

sandeep-krishnamurthy closed issue #7860: mx.nd.Custom bug
URL: https://github.com/apache/incubator-mxnet/issues/7860
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy commented on issue #7860: mx.nd.Custom bug

[GitBox]

sandeep-krishnamurthy commented on issue #7860: mx.nd.Custom bug
URL: 
https://github.com/apache/incubator-mxnet/issues/7860#issuecomment-396746895
 
 
   Resolving. Please reopen if issue still persists.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on issue #11198: [MXNET-531][Work In Progress] add Examples for Scala new API

[GitBox]

nswamy commented on issue #11198: [MXNET-531][Work In Progress] add Examples 
for Scala new API
URL: https://github.com/apache/incubator-mxnet/pull/11198#issuecomment-396744400
 
 
   @lanking520 please break into one example per PR with testing and 
documentation included


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu opened a new issue #11249: Flaky Scala test: IllegalArgumentException: requirement failed: Failed to start ps scheduler

[GitBox]

marcoabreu opened a new issue #11249: Flaky Scala test: 
IllegalArgumentException: requirement failed: Failed to start ps scheduler
URL: https://github.com/apache/incubator-mxnet/issues/11249
 
 
   
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11246/1/pipeline/
   ``Exception in thread "Thread-113" java.lang.IllegalArgumentException: 
requirement failed: Failed to start ps scheduler process with exit code 134``
   
   ```
   18/06/12 17:16:53 INFO Utils: 
/work/mxnet/scala-package/assembly/linux-x86_64-cpu/target/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar
 has been previously copied to 
/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar
   
   18/06/12 17:16:53 INFO Executor: Fetching 
file:/work/mxnet/scala-package/spark/target/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar 
with timestamp 1528823813012
   
   18/06/12 17:16:53 INFO Utils: 
/work/mxnet/scala-package/spark/target/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar has 
been previously copied to 
/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar
   
   18/06/12 17:16:53 INFO MXNet: Starting server ...
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:234881024+8528911
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:67108864+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:167772160+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:201326592+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:134217728+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:33554432+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:0+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:100663296+33554432
   
   18/06/12 17:16:53 INFO ParameterServer: Started process: java  -cp 
/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar:/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar
 org.apache.mxnet.spark.ParameterServer --role=server --root-uri=172.17.0.4 
--root-port=45669 --num-server=1 --num-worker=2 --timeout=300 at 
172.17.0.4:45669
   
   18/06/12 17:16:53 INFO ParameterServer: Starting InputStream-Redirecter 
Thread for 172.17.0.4:45669
   
   18/06/12 17:16:53 INFO ParameterServer: Starting ErrorStream-Redirecter 
Thread for 172.17.0.4:45669
   
   SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
   
   SLF4J: Defaulting to no-operation (NOP) logger implementation
   
   SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.
   
   18/06/12 17:16:53 INFO Executor: Finished task 7.0 in stage 1.0 (TID 8). 
2254 bytes result sent to driver
   
   18/06/12 17:16:53 INFO TaskSetManager: Finished task 7.0 in stage 1.0 (TID 
8) in 530 ms on localhost (1/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 
1) in 1083 ms on localhost (2/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 5.0 in stage 1.0 (TID 6). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 5.0 in stage 1.0 (TID 
6) in 1092 ms on localhost (3/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 2.0 in stage 1.0 (TID 3). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO Executor: Finished task 1.0 in stage 1.0 (TID 2). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 2.0 in stage 1.0 (TID 
3) in 1093 ms on localhost (4/8)
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 
2) in 1094 ms on localhost (5/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 6.0 in stage 1.0 (TID 7). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 6.0 in stage 1.0 (TID 
7) in 1097 ms on localhost (6/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 3.0 in stage 1.0 (TID 4). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 3.0 in stage 1.0 (TID 
4) in 1100 ms on localhost 

[GitHub] sandeep-krishnamurthy commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

sandeep-krishnamurthy commented on a change in pull request #11213: [MXNET-533] 
MXNet-ONNX export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194894587
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/export_helper.py
 ##
 @@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(sym_filepath, params_filepath, input_shape):
 
 Review comment:
   Nice corner case that is hard to think of  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194892098
 
 

 ##
 File path: python/mxnet/contrib/onnx/_export/op_translations.py
 ##
 @@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+#  https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+#  Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+#  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+#  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+#  PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+#  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+#  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+#  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+#  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"], 
kwargs["in_shape"])
+return [tval_node]
+
+
+def parse_helper(attrs, attrs_name, alt_value=None):
+"""Helper function to parse operator attributes in required format."""
+tuple_re = re.compile('\([0-9L|,| ]+\)')
+if attrs is None:
+return alt_value
+attrs_str = str(attrs.get(attrs_name))
+if attrs_str is None:
+return alt_value
+attrs_match = tuple_re.search(attrs_str)
+if attrs_match is not None:
+if attrs_match.span() == (0, len(attrs_str)):
+dims = eval(attrs_str)
+return dims
+else:
+raise AttributeError("Malformed %s dimensions: %s" % (attrs_name, 
str(attrs_str)))
+return alt_value
+
+
+@mx_op.register("Convolution")
+def convert_convolution(node, **kwargs):
+"""Map MXNet's convolution operator 

[GitHub] marcoabreu commented on a change in pull request #11245: [MXNET-244][MXNET-523][ARM] improvements to ARMv7 based builds.

[GitBox]

marcoabreu commented on a change in pull request #11245: 
[MXNET-244][MXNET-523][ARM] improvements to ARMv7 based builds.
URL: https://github.com/apache/incubator-mxnet/pull/11245#discussion_r194890988
 
 

 ##
 File path: tests/python/unittest/test_ndarray.py
 ##
 @@ -711,9 +711,8 @@ def get_values(ensure_unique):
  k=dat_size*dat_size*dat_size*dat_size, is_ascend=False)
 assert_almost_equal(nd_ret_argsort, gt)
 
-# test topk with a big shape
-a = mx.nd.arange(0, 54686454, step=1, repeat=1)
-assert_almost_equal(a.topk(k=54686454).asnumpy(), a.asnumpy()[::-1])
+a = mx.nd.arange(0, 1024, step=1, repeat=1)
+assert_almost_equal(a.topk(k=1024).asnumpy(), a.asnumpy()[::-1])
 
 Review comment:
   As stated before, leave this test as is. You're defeating the purpose of 
this test with this modification.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11226: Fix build.py when CCACHE_DIR is set.

[GitBox]

marcoabreu commented on issue #11226: Fix build.py when CCACHE_DIR is set.
URL: https://github.com/apache/incubator-mxnet/pull/11226#issuecomment-396735496
 
 
   Yes I tried locally, see 
https://github.com/apache/incubator-mxnet/blob/master/ci/test_docker_cache.py 
for reference.
   This is needed because by default Docker is not going to use untrusted cache 
layers - including the ones retrieved via docker pull. This argument disables 
this security measurement to allow untrusted cache layers to be considered.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #62: Seed files for broken link checker job

[GitBox]

marcoabreu commented on issue #62: Seed files for broken link checker job
URL: 
https://github.com/apache/incubator-mxnet-site/pull/62#issuecomment-396734859
 
 
   Thank you. 
   
   Please work together with Aaron to get this merged. His script automatically 
purges all content of this repository, so he will have to add an exception for 
this directory.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

marcoabreu commented on a change in pull request #11126: [MXNET-386] ongoing 
maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126#discussion_r194889202
 
 

 ##
 File path: 
scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
 ##
 @@ -97,9 +97,11 @@ private[mxnet] object APIDocGenerator{
   argDef += "name : String = null"
   argDef += "attr : Map[String, String] = null"
 } else {
+  argDef += "out : Option[NDArray] = None"
   returnType = "org.apache.mxnet.NDArrayFuncReturn"
 }
-s"def ${func.name} (${argDef.mkString(", ")}) : ${returnType}"
+val experimentalTag = "@Experimental"
 
 Review comment:
   Oh I see, I'm sorry. I was under the impression that it was similar to 
https://www.scala-lang.org/files/archive/nightly/docs/library/index.html#scala.deprecated
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

Roshrini commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX 
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194885045
 
 

 ##
 File path: python/mxnet/contrib/onnx/_import/op_translations.py
 ##
 @@ -114,7 +114,7 @@ def maximum(attrs, inputs, proto_obj):
 for op_input in inputs[2:]:
 mxnet_op = symbol.maximum(mxnet_op, op_input)
 else:
-mxnet_op = inputs[0]
+mxnet_op = symbol.maximum(inputs[0], inputs[0])
 
 Review comment:
   yes, onnx has a case where if there is only one input, it returns that input 
itself as output. MXNet needs 2 inputs always


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy closed pull request #11140: [MXNET-344] [ONNX-MXNet] Add new Operator Translations for ONNX import module

[GitBox]

sandeep-krishnamurthy closed pull request #11140: [MXNET-344] [ONNX-MXNet] Add 
new Operator Translations for ONNX import module
URL: https://github.com/apache/incubator-mxnet/pull/11140
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/mxnet/contrib/onnx/_import/import_helper.py 
b/python/mxnet/contrib/onnx/_import/import_helper.py
index c8d45216729..3dfff3ed681 100644
--- a/python/mxnet/contrib/onnx/_import/import_helper.py
+++ b/python/mxnet/contrib/onnx/_import/import_helper.py
@@ -32,6 +32,10 @@
 from .op_translations import reduce_max, reduce_mean, reduce_min, reduce_sum
 from .op_translations import reduce_prod, avg_pooling, max_pooling
 from .op_translations import argmax, argmin, maximum, minimum
+from .op_translations import clip, reduce_log_sum, reduce_log_sum_exp
+from .op_translations import reduce_sum_square, reduce_l2, max_roi_pooling, 
instance_norm
+from .op_translations import log_softmax, softsign, lesser, greater, equal
+from .op_translations import logical_and, logical_or, logical_xor, logical_not
 
 # convert_map defines maps of ONNX operator names to converter 
functor(callable)
 # defined in the op_translations module.
@@ -102,6 +106,22 @@
 # Sorting and Searching
 'ArgMax': argmax,
 'ArgMin': argmin,
-'Max'   : maximum, #elemwise maximum
-'Min'   : minimum #elemwise minimum
+'Max'   : maximum,
+'Min'   : minimum,
+'Clip'  : clip,
+'ReduceLogSum'  : reduce_log_sum,
+'ReduceLogSumExp'   : reduce_log_sum_exp,
+'ReduceSumSquare'   : reduce_sum_square,
+'ReduceL2'  : reduce_l2,
+'MaxRoiPool': max_roi_pooling,
+'InstanceNormalization' : instance_norm,
+'LogSoftmax': log_softmax,
+'Softsign'  : softsign,
+'Less'  : lesser,
+'Greater'   : greater,
+'Equal' : equal,
+'And'   : logical_and,
+'Xor'   : logical_xor,
+'Not'   : logical_not,
+'Or': logical_or
 }
diff --git a/python/mxnet/contrib/onnx/_import/op_translations.py 
b/python/mxnet/contrib/onnx/_import/op_translations.py
index e02cb0c2b62..0fad0080bef 100644
--- a/python/mxnet/contrib/onnx/_import/op_translations.py
+++ b/python/mxnet/contrib/onnx/_import/op_translations.py
@@ -18,6 +18,7 @@
 # coding: utf-8
 """ Module for translating ONNX operators into Mxnet operatoes"""
 # pylint: disable=unused-argument,protected-access
+import numpy as np
 from . import translation_utils
 from  import symbol
 
@@ -80,6 +81,22 @@ def divide(attrs, inputs, proto_obj):
 return op_value, new_attr, inputs
 return 'broadcast_div', new_attr, inputs
 
+def logical_and(attrs, inputs, proto_obj):
+"""Logical and of two input arrays."""
+return 'broadcast_logical_and', attrs, inputs
+
+def logical_or(attrs, inputs, proto_obj):
+"""Logical or of two input arrays."""
+return 'broadcast_logical_or', attrs, inputs
+
+def logical_xor(attrs, inputs, proto_obj):
+"""Logical xor of two input arrays."""
+return 'broadcast_logical_xor', attrs, inputs
+
+def logical_not(attrs, inputs, proto_obj):
+"""Logical not of two input arrays."""
+return 'logical_not', attrs, inputs
+
 def absolute(attrs, inputs, proto_obj):
 """Returns element-wise absolute value of the input."""
 return 'abs', attrs, inputs
@@ -97,7 +114,6 @@ def argmax(attrs, inputs, proto_obj):
 """Returns indices of the maximum values along an axis"""
 return 'argmax', attrs, inputs
 
-
 def argmin(attrs, inputs, proto_obj):
 """Returns indices of the minimum values along an axis."""
 return 'argmin', attrs, inputs
@@ -130,6 +146,18 @@ def minimum(attrs, inputs, proto_obj):
 mxnet_op = inputs[0]
 return mxnet_op, attrs, inputs
 
+def lesser(attrs, inputs, proto_obj):
+"""Logical Lesser operator with broadcasting."""
+return 'broadcast_lesser', attrs, inputs
+
+def greater(attrs, inputs, proto_obj):
+"""Logical Greater operator with broadcasting."""
+return 'broadcast_greater', attrs, inputs
+
+def equal(attrs, inputs, proto_obj):
+"""Logical Equal operator with broadcasting."""
+return 'broadcast_equal', attrs, inputs
+
 #Hyperbolic functions
 def tanh(attrs, inputs, proto_obj):
 """Returns the hyperbolic tangent of the input array."""
@@ -151,6 +179,10 @@ def concat(attrs, inputs, proto_obj):
 return 'concat', new_attrs, inputs
 
 # Basic neural network functions
+def softsign(attrs, inputs, proto_obj):
+"""Computes softsign of x element-wise."""
+return 'softsign', attrs, inputs
+
 def sigmoid(attrs, inputs, proto_obj):
 """Computes elementwise 

[GitHub] ThomasDelteil commented on a change in pull request #11210: [MXNET-532] Clarify documentation of save_params(), load_params(), export()

[GitBox]

ThomasDelteil commented on a change in pull request #11210: [MXNET-532] Clarify 
documentation of save_params(), load_params(), export()
URL: https://github.com/apache/incubator-mxnet/pull/11210#discussion_r194880263
 
 

 ##
 File path: python/mxnet/gluon/block.py
 ##
 @@ -309,6 +309,17 @@ def _collect_params_with_prefix(self, prefix=''):
 
 def save_params(self, filename):
 """Save parameters to file.
+This function is to be used to save parameters of a Gluon model, note 
that
+the saved parameters are not meant to be loaded in a different 
language binding for now.
+Saving parameters using `.save_params()` is different than
+`.collect_params().save()`, which is a deprecated way to save 
parameters of a model
+and should be avoided.
 
 Review comment:
   But as it stands against current master and current save_params and 
load_params, this documentation is valid


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudhacharya commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export

[GitBox]

anirudhacharya commented on a change in pull request #11213: [MXNET-533] 
MXNet-ONNX export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194879799
 
 

 ##
 File path: python/mxnet/contrib/onnx/__init__.py
 ##
 @@ -18,3 +18,4 @@
 
 from ._import.import_model import import_model, get_model_metadata
 
 Review comment:
   ``import`` is a reserved keyword, we cant have a folder called import. we 
can probably rename the two folders to onnx_import and onnx_export and make its 
member files private, except for the modules that we are exposing to the user.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on a change in pull request #11210: [MXNET-532] Clarify documentation of save_params(), load_params(), export()

[GitBox]

ThomasDelteil commented on a change in pull request #11210: [MXNET-532] Clarify 
documentation of save_params(), load_params(), export()
URL: https://github.com/apache/incubator-mxnet/pull/11210#discussion_r194879113
 
 

 ##
 File path: python/mxnet/gluon/block.py
 ##
 @@ -309,6 +309,17 @@ def _collect_params_with_prefix(self, prefix=''):
 
 def save_params(self, filename):
 """Save parameters to file.
+This function is to be used to save parameters of a Gluon model, note 
that
+the saved parameters are not meant to be loaded in a different 
language binding for now.
+Saving parameters using `.save_params()` is different than
+`.collect_params().save()`, which is a deprecated way to save 
parameters of a model
+and should be avoided.
 
 Review comment:
   no I don't agree. I think the documentation, as written here is valid and 
should be moved to `save_parameters` and `load_parameters`.
   
   ```
   Saving parameters using `.save_params()` is different than
   +`.collect_params().save()`, which is a deprecated way to save 
parameters of a model
   +and should be avoided.
   ```
   ->
   ```
   Saving parameters using `.save_parameters()` is different than
   +`.collect_params().save()` and .save_params() which are deprecated 
ways to save parameters of a model
   +and should be avoided.
   ```
   Once the other PR is merged


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on a change in pull request #11210: [MXNET-532] Clarify documentation of save_params(), load_params(), export()

[GitBox]

ThomasDelteil commented on a change in pull request #11210: [MXNET-532] Clarify 
documentation of save_params(), load_params(), export()
URL: https://github.com/apache/incubator-mxnet/pull/11210#discussion_r194879113
 
 

 ##
 File path: python/mxnet/gluon/block.py
 ##
 @@ -309,6 +309,17 @@ def _collect_params_with_prefix(self, prefix=''):
 
 def save_params(self, filename):
 """Save parameters to file.
+This function is to be used to save parameters of a Gluon model, note 
that
+the saved parameters are not meant to be loaded in a different 
language binding for now.
+Saving parameters using `.save_params()` is different than
+`.collect_params().save()`, which is a deprecated way to save 
parameters of a model
+and should be avoided.
 
 Review comment:
   no I don't agree. I think the documentation here is the one that should be 
moved to save_parameters and load_parameters.
   
   ```
   Saving parameters using `.save_params()` is different than
   +`.collect_params().save()`, which is a deprecated way to save 
parameters of a model
   +and should be avoided.
   ```
   ->
   ```
   Saving parameters using `.save_parameters()` is different than
   +`.collect_params().save()` and .save_params() which are deprecated 
ways to save parameters of a model
   +and should be avoided.
   ```
   Once the other PR is merged


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services