[GitHub] wkcn commented on issue #9939: add multi proposal operator (cpu version)

[GitBox]

wkcn commented on issue #9939: add multi proposal operator (cpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#issuecomment-369499574
 
 
   @TaoLv Thank you!
   It seems that there is no any unit-test for Proposal Operator. But the test 
of Multi-Proposal relies on that of Proposal.
   Which file I should add an unit-test for the Multi-Proposal operator into?
   Is it enough to use the testing code I wrote in Comments?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yajiedesign opened a new pull request #9940: remove set CMAKE_GENERATOR_TOOLSET

[GitBox]

yajiedesign opened a new pull request #9940: remove set CMAKE_GENERATOR_TOOLSET
URL: https://github.com/apache/incubator-mxnet/pull/9940
 
 
   ## Description ##
   remove set CMAKE_GENERATOR_TOOLSET,This parameter should be set up by -T
   
   ## Checklist ##
   ### Essentials ###
   - [x] Passed code style checking (`make lint`)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171477959
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -237,13 +237,14 @@ mkldnn_memory_format_t 
GetDefaultFormat(mkldnn::memory::desc desc) {
   case mkldnn_gOIhw16o16i:
   case mkldnn_gIOhw16o16i:
   case mkldnn_gOihw8o:
+  case mkldnn_Goihw8g:
   case mkldnn_gOihw16o:
   case mkldnn_gOhwi8o:
   case mkldnn_gOhwi16o:
   case mkldnn_gOhIw16o4i:
 return mkldnn_goihw;
   default:
-LOG(FATAL) << "Unknown MKLDNN format for 4 dimensions: " << 
desc.data.format;
+LOG(FATAL) << "Unknown MKLDNN format for 5 dimensions: " << 
desc.data.format;
 
 Review comment:
   Yes


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] TaoLv commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

TaoLv commented on a change in pull request #9918: Update mkldnn to the newest 
& Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171476701
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -237,13 +237,14 @@ mkldnn_memory_format_t 
GetDefaultFormat(mkldnn::memory::desc desc) {
   case mkldnn_gOIhw16o16i:
   case mkldnn_gIOhw16o16i:
   case mkldnn_gOihw8o:
+  case mkldnn_Goihw8g:
   case mkldnn_gOihw16o:
   case mkldnn_gOhwi8o:
   case mkldnn_gOhwi16o:
   case mkldnn_gOhIw16o4i:
 return mkldnn_goihw;
   default:
-LOG(FATAL) << "Unknown MKLDNN format for 4 dimensions: " << 
desc.data.format;
+LOG(FATAL) << "Unknown MKLDNN format for 5 dimensions: " << 
desc.data.format;
 
 Review comment:
   Do you mean a negative test case which can run into the default statement?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171475825
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -237,13 +237,14 @@ mkldnn_memory_format_t 
GetDefaultFormat(mkldnn::memory::desc desc) {
   case mkldnn_gOIhw16o16i:
   case mkldnn_gIOhw16o16i:
   case mkldnn_gOihw8o:
+  case mkldnn_Goihw8g:
   case mkldnn_gOihw16o:
   case mkldnn_gOhwi8o:
   case mkldnn_gOhwi16o:
   case mkldnn_gOhIw16o4i:
 return mkldnn_goihw;
   default:
-LOG(FATAL) << "Unknown MKLDNN format for 4 dimensions: " << 
desc.data.format;
+LOG(FATAL) << "Unknown MKLDNN format for 5 dimensions: " << 
desc.data.format;
 
 Review comment:
   Aah great. But the error message is indeed pretty misleading. Would it be 
possible to make a better fitting test?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] TaoLv commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

TaoLv commented on a change in pull request #9918: Update mkldnn to the newest 
& Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171473792
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -237,13 +237,14 @@ mkldnn_memory_format_t 
GetDefaultFormat(mkldnn::memory::desc desc) {
   case mkldnn_gOIhw16o16i:
   case mkldnn_gIOhw16o16i:
   case mkldnn_gOihw8o:
+  case mkldnn_Goihw8g:
   case mkldnn_gOihw16o:
   case mkldnn_gOhwi8o:
   case mkldnn_gOhwi16o:
   case mkldnn_gOhIw16o4i:
 return mkldnn_goihw;
   default:
-LOG(FATAL) << "Unknown MKLDNN format for 4 dimensions: " << 
desc.data.format;
+LOG(FATAL) << "Unknown MKLDNN format for 5 dimensions: " << 
desc.data.format;
 
 Review comment:
   We notice that there is a test case failed here and it was caused by a 
missing data format "mkldnn_Goihw8g". Then we found the error message is 
misleading.
   Please find the error message here: 
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-9918/4/pipeline/513


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] TaoLv commented on issue #9939: add multi proposal operator (cpu version)

[GitBox]

TaoLv commented on issue #9939: add multi proposal operator (cpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#issuecomment-369487715
 
 
   Thanks for adding the CPU implementation.. Would you like to add test cases 
for it, so the operator can be checked in jenkins.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171472298
 
 

 ##
 File path: tests/ci_build/Dockerfile.ubuntu1404_cuda75_cudnn5
 ##
 @@ -38,3 +38,9 @@ RUN add-apt-repository -y ppa:marutter/rdev
 RUN apt-get update && apt-get -y upgrade
 RUN DEBIAN_FRONTEND=noninteractive apt-get -y -o 
Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confnew" install 
r-base r-base-dev
 RUN Rscript -e "install.packages('devtools', repo = 
'https://cran.rstudio.com')"
+
+# Add MKLML library, compatiable with Ubuntu16.04
 
 Review comment:
   Remove


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171472359
 
 

 ##
 File path: tests/ci_build/Dockerfile.amzn_linux_cpu
 ##
 @@ -14,3 +14,9 @@ RUN /install/install_testdeps.sh
 RUN /install/install_julia.sh
 RUN /install/install_maven.sh
 RUN /install/install_library.sh
+
+# Add MKLML library, compatiable with Ubuntu16.04
 
 Review comment:
   Remove


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171472328
 
 

 ##
 File path: tests/ci_build/Dockerfile.cpu
 ##
 @@ -10,3 +10,9 @@ COPY install/ubuntu_install_r.sh /install/
 RUN /install/ubuntu_install_r.sh
 COPY install/ubuntu_install_perl.sh /install/
 RUN /install/ubuntu_install_perl.sh
+
+# Add MKLML library, compatiable with Ubuntu16.04
 
 Review comment:
   Remove


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171472242
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -237,13 +237,14 @@ mkldnn_memory_format_t 
GetDefaultFormat(mkldnn::memory::desc desc) {
   case mkldnn_gOIhw16o16i:
   case mkldnn_gIOhw16o16i:
   case mkldnn_gOihw8o:
+  case mkldnn_Goihw8g:
   case mkldnn_gOihw16o:
   case mkldnn_gOhwi8o:
   case mkldnn_gOhwi16o:
   case mkldnn_gOhIw16o4i:
 return mkldnn_goihw;
   default:
-LOG(FATAL) << "Unknown MKLDNN format for 4 dimensions: " << 
desc.data.format;
+LOG(FATAL) << "Unknown MKLDNN format for 5 dimensions: " << 
desc.data.format;
 
 Review comment:
   Is there a test for this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on issue #9918: Update mkldnn to the newest & Add clang 
build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#issuecomment-369487211
 
 
   Please only UPDATE dockerfiles that already have mkl stuff in it. Don't add 
it unless necessary.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pengzhao-intel commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

pengzhao-intel commented on a change in pull request #9918: Update mkldnn to 
the newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171471941
 
 

 ##
 File path: tests/ci_build/Dockerfile.amzn_linux_cpu
 ##
 @@ -14,3 +14,9 @@ RUN /install/install_testdeps.sh
 RUN /install/install_julia.sh
 RUN /install/install_maven.sh
 RUN /install/install_library.sh
+
+# Add MKLML library, compatiable with Ubuntu16.04
+RUN wget --no-check-certificate -O /tmp/mklml.tgz 
https://github.com/01org/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz
 
 Review comment:
   MKL-DNN repo is moved to github/intel/mkl-dnn. Please use the new repo :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wkcn opened a new pull request #9939: add multi proposal operator (cpu version)

[GitBox]

wkcn opened a new pull request #9939: add multi proposal operator (cpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939
 
 
   ## Description ##
   The multi_proposal operator (`mxnet.sym.contrib.MultiProposal`, CPU version) 
is not implemented before.
   I wrote the code about it.
   I didn't change the author list. Could I add my name on it?
   
   ## Checklist ##
   ### Essentials ###
   - [x] Passed code style checking (`make lint`)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] Multi Proposal Operator is supported on the CPU context.
   
   ## Comments ##
   Here is the testing code.
   ```python
   import mxnet as mx
   from mxnet import nd
   import numpy as np
   
   feature_stride = 16 
   scales = (8, 16, 32)
   ratios = (0.5, 1, 2)
   rpn_pre_nms_top_n = 12000 
   rpn_post_nms_top_n = 2000
   threshold = 0.7
   rpn_min_size = 16
   
   batch_size = 20
   feat_len = 14
   H, W = feat_len, feat_len
   num_anchors = len(scales) * len(ratios)
   count_anchors = feat_len * feat_len * num_anchors
   
   '''
   cls_prob: (batch_size, 2 * num_anchors, H, W)
   bbox_pred: (batch_size, 4 * num_anchors, H, W) 
   im_info: (batch_size, 3)
   '''
   
   cls_prob = nd.empty((batch_size, 2 * num_anchors, H, W), dtype = np.float32) 
   bbox_pred = nd.empty((batch_size, 4 * num_anchors, H, W), dtype = 
np.float32) 
   im_info = nd.empty((batch_size, 3), dtype = np.float32) 
   
   cls_prob = nd.array(np.random.random(cls_prob.shape))
   bbox_pred = nd.array(np.random.random(bbox_pred.shape))
   
   for i in range(batch_size):
   im_size = np.random.randint(100, feat_len * feature_stride, size = (2,))
   im_scale = np.random.randint(70, 100) / 100.0
   im_info[i, :] = [im_size[0], im_size[1], im_scale]
   
   def get_sub(arr, i):
   new_shape = list(arr.shape)
   new_shape[0] = 1
   res = arr[i].reshape(new_shape)
   return res
   
   def test(rpn_pre_nms_top_n, rpn_post_nms_top_n):
   
   single_proposal = []
   single_score = []
   for i in range(batch_size):
   rois, score = mx.nd.contrib.Proposal(
   cls_score = get_sub(cls_prob, i),
   bbox_pred = get_sub(bbox_pred, i),
   im_info = get_sub(im_info, i),
   feature_stride = feature_stride,
   scales = scales,
   ratios = ratios,
   rpn_pre_nms_top_n = rpn_pre_nms_top_n,
   rpn_post_nms_top_n = rpn_post_nms_top_n,
   threshold = threshold,
   rpn_min_size = rpn_min_size, output_score = True)
   single_proposal.append(rois)
   single_score.append(score)
   
   multi_proposal, multi_score = mx.nd.contrib.MultiProposal(
   cls_score = cls_prob,
   bbox_pred = bbox_pred,
   im_info = im_info,
   feature_stride = feature_stride,
   scales = scales,
   ratios = ratios,
   rpn_pre_nms_top_n = rpn_pre_nms_top_n,
   rpn_post_nms_top_n = rpn_post_nms_top_n,
   threshold = threshold,
   rpn_min_size = rpn_min_size, output_score = True)
   
   single_proposal = 
nd.stack(*single_proposal).reshape(multi_proposal.shape)
   single_score = nd.stack(*single_score).reshape(multi_score.shape)
   
   single_proposal_np = single_proposal.asnumpy()
   multi_proposal_np = multi_proposal.asnumpy()
   
   single_score_np = single_score.asnumpy()
   multi_score_np = multi_score.asnumpy()
   print (multi_proposal_np.shape)
   
   # test rois x1,y1,x2,y2
   assert np.allclose(single_proposal_np[:, 1:], multi_proposal_np[:, 1:])
   # test rois batch_idx 
   for i in range(batch_size):
   start = i * rpn_post_nms_top_n
   end = start + rpn_post_nms_top_n
   assert (multi_proposal_np[start:end, 0] == i).all()
   
   # test score
   assert np.allclose(single_score_np, multi_score_np)
   
   test(rpn_pre_nms_top_n, rpn_post_nms_top_n)
   

[GitHub] reminisce commented on issue #9937: fatal error: cblas.h: No such file or directory

[GitBox]

reminisce commented on issue #9937: fatal error: cblas.h: No such file or 
directory
URL: 
https://github.com/apache/incubator-mxnet/issues/9937#issuecomment-369483551
 
 
   You need to install openblas. Please follow the steps here for building from 
source.
   http://mxnet.incubator.apache.org/install/index.html
   Try this 
   `sudo apt-get install libopenblas-dev`.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pengzhao-intel commented on issue #9923: Removed MXNET_USE_MKLDNN definition when option USE_MKLDNN is disabled

[GitBox]

pengzhao-intel commented on issue #9923: Removed MXNET_USE_MKLDNN definition 
when option USE_MKLDNN is disabled
URL: https://github.com/apache/incubator-mxnet/pull/9923#issuecomment-369479074
 
 
   LGTM :+1: 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sxjscience commented on issue #9934: [Gluon] Support variant sequence length in gluon.RecurrentCell

[GitBox]

sxjscience commented on issue #9934: [Gluon] Support variant sequence length in 
gluon.RecurrentCell 
URL: https://github.com/apache/incubator-mxnet/pull/9934#issuecomment-369473762
 
 
   The failing message is very strange. Only fails on two machines.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: fix word_language_model for python3 (#9926)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new e29b2f5  fix word_language_model for python3 (#9926)
e29b2f5 is described below

commit e29b2f5bcac0f1cefaad9232879e7165fa51f67d
Author: Xingjian Shi 
AuthorDate: Wed Feb 28 20:48:56 2018 -0800

fix word_language_model for python3 (#9926)
---
 example/gluon/word_language_model/train.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/example/gluon/word_language_model/train.py 
b/example/gluon/word_language_model/train.py
index 001e9f4..b69fd17 100644
--- a/example/gluon/word_language_model/train.py
+++ b/example/gluon/word_language_model/train.py
@@ -78,21 +78,21 @@ val_dataset, test_dataset = 
[contrib.data.text.WikiText2('./data', segment,
  seq_len=args.bptt)
  for segment in ['validation', 'test']]
 
-nbatch_train = len(train_dataset) / args.batch_size
+nbatch_train = len(train_dataset) // args.batch_size
 train_data = gluon.data.DataLoader(train_dataset,
batch_size=args.batch_size,

sampler=contrib.data.IntervalSampler(len(train_dataset),
 
nbatch_train),
last_batch='discard')
 
-nbatch_val = len(val_dataset) / args.batch_size
+nbatch_val = len(val_dataset) // args.batch_size
 val_data = gluon.data.DataLoader(val_dataset,
  batch_size=args.batch_size,
  
sampler=contrib.data.IntervalSampler(len(val_dataset),
   
nbatch_val),
  last_batch='discard')
 
-nbatch_test = len(test_dataset) / args.batch_size
+nbatch_test = len(test_dataset) // args.batch_size
 test_data = gluon.data.DataLoader(test_dataset,
   batch_size=args.batch_size,
   
sampler=contrib.data.IntervalSampler(len(test_dataset),

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] szha closed pull request #9926: Fix Gluon Language Model for python3

[GitBox]

szha closed pull request #9926: Fix Gluon Language Model for python3
URL: https://github.com/apache/incubator-mxnet/pull/9926
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/example/gluon/word_language_model/train.py 
b/example/gluon/word_language_model/train.py
index 001e9f4930e..b69fd173200 100644
--- a/example/gluon/word_language_model/train.py
+++ b/example/gluon/word_language_model/train.py
@@ -78,21 +78,21 @@
  seq_len=args.bptt)
  for segment in ['validation', 'test']]
 
-nbatch_train = len(train_dataset) / args.batch_size
+nbatch_train = len(train_dataset) // args.batch_size
 train_data = gluon.data.DataLoader(train_dataset,
batch_size=args.batch_size,

sampler=contrib.data.IntervalSampler(len(train_dataset),
 
nbatch_train),
last_batch='discard')
 
-nbatch_val = len(val_dataset) / args.batch_size
+nbatch_val = len(val_dataset) // args.batch_size
 val_data = gluon.data.DataLoader(val_dataset,
  batch_size=args.batch_size,
  
sampler=contrib.data.IntervalSampler(len(val_dataset),
   
nbatch_val),
  last_batch='discard')
 
-nbatch_test = len(test_dataset) / args.batch_size
+nbatch_test = len(test_dataset) // args.batch_size
 test_data = gluon.data.DataLoader(test_dataset,
   batch_size=args.batch_size,
   
sampler=contrib.data.IntervalSampler(len(test_dataset),


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #9938: mxnet installation issue on Win 10 - module gluon.data.dataset has no attribute DownloadedDataset

[GitBox]

szha commented on issue #9938: mxnet installation issue on Win 10 - module 
gluon.data.dataset has no attribute DownloadedDataset
URL: 
https://github.com/apache/incubator-mxnet/issues/9938#issuecomment-369462439
 
 
   @yajiedesign 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KishoreKarunakaran opened a new issue #9938: mxnet installation issue on Win 10

[GitBox]

KishoreKarunakaran opened a new issue #9938: mxnet installation issue on Win 10
URL: https://github.com/apache/incubator-mxnet/issues/9938
 
 
   I have used **pip install mxnet** to install mxnet and now I'm getting an 
**AttributeError: module 'mxnet.gluon.data.dataset' has no attribute 
'_DownloadedDataset'**
   
   Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 10:22:32) [MSC v.1900 
64 bit (AMD64)] on win32
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import mxnet as ms
   E:\Programs\Anaconda3\envs\mxnet\lib\site-packages\h5py\__init__.py:36: 
FutureWarning: Conversion of the second argument of issubdtype from `float` to 
`np.floating` is deprecated. In future, it will be treated as `np.float64 == 
np.dtype(float).type`.
 from ._conv import register_converters as _register_converters
   Traceback (most recent call last):
 File "", line 1, in 
 File 
"E:\Programs\Anaconda3\envs\mxnet\lib\site-packages\mxnet\__init__.py", line 
81, in 
   from . import gluon
 File 
"E:\Programs\Anaconda3\envs\mxnet\lib\site-packages\mxnet\gluon\__init__.py", 
line 36, in 
   from . import data
 File 
"E:\Programs\Anaconda3\envs\mxnet\lib\site-packages\mxnet\gluon\data\__init__.py",
 line 28, in 
   from . import vision
 File 
"E:\Programs\Anaconda3\envs\mxnet\lib\site-packages\mxnet\gluon\data\vision\__init__.py",
 line 22, in 
   from .datasets import *
 File 
"E:\Programs\Anaconda3\envs\mxnet\lib\site-packages\mxnet\gluon\data\vision\datasets.py",
 line 36, in 
   class MNIST(dataset._DownloadedDataset):
   AttributeError: module 'mxnet.gluon.data.dataset' has no attribute 
'_DownloadedDataset'
   >>>
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhengduoru opened a new issue #9937: fatal error: cblas.h: No such file or directory

[GitBox]

zhengduoru opened a new issue #9937: fatal error: cblas.h: No such file or 
directory
URL: https://github.com/apache/incubator-mxnet/issues/9937
 
 
   My system is Ubuntu 16.04. 
   when I run code `make -j4  USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1 
USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1`
   
   I got the error 
   
   
   g++ -std=c++11 -c -DMSHADOW_FORCE_STREAM -Wall -Wsign-compare -O3 
-I/home/duoduo/github/mxnet/mshadow/ 
-I/home/duoduo/github/mxnet/dmlc-core/include -fPIC 
-I/home/duoduo/github/mxnet/nnvm/include -Iinclude -funroll-loops 
-Wno-unused-variable -Wno-unused-parameter -Wno-unknown-pragmas 
-Wno-unused-local-typedefs -msse3 -I/usr/local/cuda/include 
-DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 
-DMSHADOW_DIST_PS=0 -DMSHADOW_USE_PASCAL=0 -DMXNET_USE_OPENCV=1 
-I/usr/local/include/opencv -I/usr/local/include -fopenmp -DMSHADOW_USE_CUDNN=1 
 -I/home/duoduo/github/mxnet/cub -DMXNET_USE_NVRTC=0 -MMD -c 
src/operator/nn/softmax.cc -o build/src/operator/nn/softmax.o
   g++ -std=c++11 -c -DMSHADOW_FORCE_STREAM -Wall -Wsign-compare -O3 
-I/home/duoduo/github/mxnet/mshadow/ 
-I/home/duoduo/github/mxnet/dmlc-core/include -fPIC 
-I/home/duoduo/github/mxnet/nnvm/include -Iinclude -funroll-loops 
-Wno-unused-variable -Wno-unused-parameter -Wno-unknown-pragmas 
-Wno-unused-local-typedefs -msse3 -I/usr/local/cuda/include 
-DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 
-DMSHADOW_DIST_PS=0 -DMSHADOW_USE_PASCAL=0 -DMXNET_USE_OPENCV=1 
-I/usr/local/include/opencv -I/usr/local/include -fopenmp -DMSHADOW_USE_CUDNN=1 
 -I/home/duoduo/github/mxnet/cub -DMXNET_USE_NVRTC=0 -MMD -c 
src/operator/mkl/mkl_memory.cc -o build/src/operator/mkl/mkl_memory.o
   g++ -std=c++11 -c -DMSHADOW_FORCE_STREAM -Wall -Wsign-compare -O3 
-I/home/duoduo/github/mxnet/mshadow/ 
-I/home/duoduo/github/mxnet/dmlc-core/include -fPIC 
-I/home/duoduo/github/mxnet/nnvm/include -Iinclude -funroll-loops 
-Wno-unused-variable -Wno-unused-parameter -Wno-unknown-pragmas 
-Wno-unused-local-typedefs -msse3 -I/usr/local/cuda/include 
-DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 
-DMSHADOW_DIST_PS=0 -DMSHADOW_USE_PASCAL=0 -DMXNET_USE_OPENCV=1 
-I/usr/local/include/opencv -I/usr/local/include -fopenmp -DMSHADOW_USE_CUDNN=1 
 -I/home/duoduo/github/mxnet/cub -DMXNET_USE_NVRTC=0 -MMD -c 
src/operator/tensor/elemwise_binary_broadcast_op_basic.cc -o 
build/src/operator/tensor/elemwise_binary_broadcast_op_basic.o
   g++ -std=c++11 -c -DMSHADOW_FORCE_STREAM -Wall -Wsign-compare -O3 
-I/home/duoduo/github/mxnet/mshadow/ 
-I/home/duoduo/github/mxnet/dmlc-core/include -fPIC 
-I/home/duoduo/github/mxnet/nnvm/include -Iinclude -funroll-loops 
-Wno-unused-variable -Wno-unused-parameter -Wno-unknown-pragmas 
-Wno-unused-local-typedefs -msse3 -I/usr/local/cuda/include 
-DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 
-DMSHADOW_DIST_PS=0 -DMSHADOW_USE_PASCAL=0 -DMXNET_USE_OPENCV=1 
-I/usr/local/include/opencv -I/usr/local/include -fopenmp -DMSHADOW_USE_CUDNN=1 
 -I/home/duoduo/github/mxnet/cub -DMXNET_USE_NVRTC=0 -MMD -c 
src/operator/tensor/elemwise_binary_op_extended.cc -o 
build/src/operator/tensor/elemwise_binary_op_extended.o
   In file included from 
/home/duoduo/github/mxnet/mshadow/mshadow/tensor.h:16:0,
from include/mxnet/base.h:13,
from src/operator/nn/./../mxnet_op.h:10,
from src/operator/nn/./softmax-inl.h:11,
from src/operator/nn/softmax.cc:6:
   /home/duoduo/github/mxnet/mshadow/mshadow/./base.h:136:23: fatal error: 
cblas.h: No such file or directory
   compilation terminated.
   Makefile:207: recipe for target 'build/src/operator/nn/softmax.o' failed
   make: *** [build/src/operator/nn/softmax.o] Error 1
   make: *** Waiting for unfinished jobs
   In file included from 
/home/duoduo/github/mxnet/mshadow/mshadow/tensor.h:16:0,
from include/mxnet/./base.h:13,
from include/mxnet/operator.h:19,
from src/operator/mkl/../operator_common.h:13,
from src/operator/mkl/mkl_memory.cc:22:
   /home/duoduo/github/mxnet/mshadow/mshadow/./base.h:136:23: fatal error: 
cblas.h: No such file or directory
   compilation terminated.
   Makefile:207: recipe for target 'build/src/operator/mkl/mkl_memory.o' failed
   make: *** [build/src/operator/mkl/mkl_memory.o] Error 1
   In file included from 
/home/duoduo/github/mxnet/mshadow/mshadow/tensor.h:16:0,
from include/mxnet/./base.h:13,
from include/mxnet/operator_util.h:24,
from src/operator/tensor/./elemwise_unary_op.h:9,
from 
src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:6:
   /home/duoduo/github/mxnet/mshadow/mshadow/./base.h:136:23: fatal error: 
cblas.h: No such file or directory
   compilation terminated.
   Makefile:207: recipe for target 

[GitHub] yajiedesign closed pull request #9798: fix cmake

[GitBox]

yajiedesign closed pull request #9798: fix cmake
URL: https://github.com/apache/incubator-mxnet/pull/9798
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 16d365355ce..ea0b3f8365b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -180,14 +180,6 @@ include_directories(${CMAKE_CURRENT_SOURCE_DIR}/src)
 if(USE_CUDA)
   find_package(CUDA REQUIRED)
   add_definitions(-DMSHADOW_USE_CUDA=1)
-  if(FIRST_CUDA AND (NOT USE_OLDCMAKECUDA))
-if(CUDA_TOOLSET STREQUAL "")
-  set(CUDA_TOOLSET "${CUDA_VERSION_STRING}")
-endif()
-set(CMAKE_GENERATOR_TOOLSET "cuda=${CUDA_TOOLSET},host=x64")
-  else()
-set(FIRST_CUDA FALSE)
-  endif()
   if(USE_NCCL)
 find_package(NCCL)
 if(NCCL_FOUND)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171450142
 
 

 ##
 File path: src/kvstore/kvstore_dist_server.h
 ##
 @@ -170,6 +187,33 @@ class KVStoreDistServer {
 app->Response(recved);
   }
 
+  void SetProfilerConfig(std::string params_str) {
+std::vector elems;
+mxnet::kvstore::split(params_str, ',', std::back_inserter(elems));
+std::vector ckeys;
+std::vector cvals;
+ckeys.reserve(elems.size());
+cvals.reserve(elems.size());
+
+for (int i=0; i < elems.size(); i++) {
+  std::vector parts;
+  mxnet::kvstore::split(elems[i], ':', std::back_inserter(parts));
+  CHECK_NOTNULL(parts[0].c_str());
+  CHECK_NOTNULL(parts[1].c_str());
+  if (parts[0] == "filename") {
+parts[1] = "rank" + std::to_string(ps::MyRank()) + "_" + parts[1];
 
 Review comment:
   c_str() will never return null


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171450417
 
 

 ##
 File path: src/kvstore/kvstore_dist_server.h
 ##
 @@ -170,6 +187,33 @@ class KVStoreDistServer {
 app->Response(recved);
   }
 
+  void SetProfilerConfig(std::string params_str) {
+std::vector elems;
+mxnet::kvstore::split(params_str, ',', std::back_inserter(elems));
+std::vector ckeys;
+std::vector cvals;
+ckeys.reserve(elems.size());
+cvals.reserve(elems.size());
+
+for (int i=0; i < elems.size(); i++) {
+  std::vector parts;
 
 Review comment:
   you never push into the vector, right?  then [0] and [1] indexing is invalid 
addressing


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jinhuang415 commented on issue #9936: Adjust MKLDNN compile flags order to resolve some corner case

[GitBox]

jinhuang415 commented on issue #9936: Adjust MKLDNN compile flags order to 
resolve some corner case
URL: https://github.com/apache/incubator-mxnet/pull/9936#issuecomment-369455915
 
 
   @pengzhao-intel add Patric


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alireza202 opened a new issue #9935: Matplotlib plot crashes Jupyter Notebook with segmentation fault: 11

[GitBox]

alireza202 opened a new issue #9935: Matplotlib plot crashes Jupyter Notebook 
with segmentation fault: 11
URL: https://github.com/apache/incubator-mxnet/issues/9935
 
 
   Note: Providing complete information in the most concise form is the best 
way to get help. This issue template serves as the checklist for essential 
information to most of the technical issues and bug reports. For non-technical 
issues and feature requests, feel free to present the information in what you 
believe is the best form.
   
   For Q & A and discussion, please start a discussion thread at 
https://discuss.mxnet.io 
   
   ## Description
   A simple plot using matplotlib in Jupyter Notebook crashes the kernel with 
the following error:
   
   ```
   Segmentation fault: 11
   
   Stack trace returned 0 entries:
   ```
   
   Note that this problem does not happen with numpy.
   
   ## Environment info (Required)
   
   ```
   What to do:
   1. Download the diagnosis script from 
https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py
   2. Run the script using `python diagnose.py` and paste its output here.
   
   --Python Info--
   ('Version  :', '2.7.14')
   ('Compiler :', 'GCC 7.2.0')
   ('Build:', ('default', 'Oct 13 2017 10:47:14'))
   ('Arch :', ('64bit', ''))
   Pip Info---
   ('Version  :', '9.0.1')
   ('Directory:', 
'/home/ghiasali/anaconda2/lib/python2.7/site-packages/pip')
   --MXNet Info---
   ('Version  :', '1.0.0')
   ('Directory:', 
'/home/ghiasali/anaconda2/lib/python2.7/site-packages/mxnet')
   Hashtag not found. Not installed from pre-built package.
   --System Info--
   ('Platform :', 
'Linux-3.2.45-0.6.wd.971.49.326.metal1.x86_64-x86_64-with-redhat-5.3-Tikanga')
   ('system   :', 'Linux')
   ('node :', 'dev-dsk-ghiasali-2a-e4b175d2.us-west-2.amazon.com')
   ('release  :', '3.2.45-0.6.wd.971.49.326.metal1.x86_64')
   ('version  :', '#1 SMP Tue Nov 15 20:09:32 UTC 2016')
   --Hardware Info--
   ('machine  :', 'x86_64')
   ('processor:', 'x86_64')
   Architecture:  x86_64
   CPU op-mode(s):32-bit, 64-bit
   Byte Order:Little Endian
   CPU(s):16
   On-line CPU(s) list:   0-7
   Off-line CPU(s) list:  8-15
   Thread(s) per core:1
   Core(s) per socket:8
   Socket(s): 1
   NUMA node(s):  1
   Vendor ID: GenuineIntel
   CPU family:6
   Model: 63
   Stepping:  2
   CPU MHz:   2900.054
   BogoMIPS:  5830.23
   Hypervisor vendor: Xen
   Virtualization type:   full
   L1d cache: 32K
   L1i cache: 32K
   L2 cache:  256K
   L3 cache:  25600K
   NUMA node0 CPU(s): 0-7
   --Network Test--
   Setting timeout: 10
   Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0014 
sec, LOAD: 0.4869 sec.
   Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0203 sec, LOAD: 
0.1221 sec.
   Timing for FashionMNIST: 
https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz,
 DNS: 0.0217 sec, LOAD: 0.2004 sec.
   Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0122 sec, 
LOAD: 0.0699 sec.
   Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0497 sec, LOAD: 
0.0804 sec.
   Error open Gluon Tutorial(cn): https://zh.gluon.ai, , DNS 
finished in 0.0143220424652 sec.
   ```
   
   Jupyter Notebook info:
   ```
   Server Information:
   
   You are using Jupyter notebook.
   
   The version of the notebook server is 5.0.0 and is running on:
   Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40) 
   [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
   
   Current Kernel Information:
   
   Python 2.7.14 |Anaconda custom (64-bit)| (default, Oct 13 2017, 10:47:14) 
   Type "copyright", "credits" or "license" for more information.
   
   IPython 5.1.0 -- An enhanced Interactive Python.
   ? -> Introduction and overview of IPython's features.
   %quickref -> Quick reference.
   help  -> Python's own help system.
   object?   -> Details about 'object', use 'object??' for extra details.
   ```
   
   
   Package used (Python/R/Scala/Julia):
   Python: mxnet, matplotlib
   
   For Scala user, please provide:
   1. Java version: (`java -version`)
   2. Maven version: (`mvn -version`)
   3. Scala runtime if applicable: (`scala -version`)
   
   For R user, please provide R `sessionInfo()`:
   
   ## Build info (Required if built from source)
   
   Compiler (gcc/clang/mingw/visual studio):
   
   MXNet commit hash:
   (Paste the output of `git rev-parse HEAD` here.)
   
   Build config:
   (Paste the content of config.mk, or the build command.)
   
   ## Error Message:
   (Paste the complete error message, including stack 

[GitHub] jinhuang415 opened a new pull request #9936: Adjust MKLDNN compile flags order to resolve some corner case

[GitBox]

jinhuang415 opened a new pull request #9936: Adjust MKLDNN compile flags order 
to resolve some corner case
URL: https://github.com/apache/incubator-mxnet/pull/9936
 
 
   ## Description ##
   In some corner cases, the MKLDNN library link path (-L$(MKLDNNROOT)/lib, 
this could be a local path) will not be used but some system path (like 
/usr/local/lib, get via "pkg-config --libs opencv", this path may contain some 
stale mkldnn library) will be used first, in such case, mxnet/im2rec would be 
linked to unexpected mkldnn library and will cause issue. The fix is to move 
MKLDNN link path settings ahead before opencv so local MKLDNN library path will 
be used first during link time.
   
   ## Checklist ##
   ### Essentials ###
   - [x] Passed code style checking (`make lint`)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171448085
 
 

 ##
 File path: src/kvstore/kvstore_dist_server.h
 ##
 @@ -170,6 +187,33 @@ class KVStoreDistServer {
 app->Response(recved);
   }
 
+  void SetProfilerConfig(std::string params_str) {
+std::vector elems;
+mxnet::kvstore::split(params_str, ',', std::back_inserter(elems));
+std::vector ckeys;
+std::vector cvals;
+ckeys.reserve(elems.size());
+cvals.reserve(elems.size());
+
+for (int i=0; i < elems.size(); i++) {
+  std::vector parts;
+  mxnet::kvstore::split(elems[i], ':', std::back_inserter(parts));
+  CHECK_NOTNULL(parts[0].c_str());
+  CHECK_NOTNULL(parts[1].c_str());
+  if (parts[0] == "filename") {
+parts[1] = "rank" + std::to_string(ps::MyRank()) + "_" + parts[1];
+  }
+  char * ckey = new char[parts[0].length() + 1];
+  std::sprintf(ckey, "%s", parts[0].c_str());
+  ckeys.push_back(ckey);
+
+  char* cval = new char[parts[1].length() + 1];
+  std::sprintf(cval, "%s", parts[1].c_str());
 
 Review comment:
   why sprintf? it?s not very efficient compared to a strcpy, which is 
generally pretty fast 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171447567
 
 

 ##
 File path: src/kvstore/kvstore_dist.h
 ##
 @@ -92,6 +92,14 @@ class KVStoreDist : public KVStoreLocal {
 }
   }
 
+  void SetServerProfilerCommand(KVStoreServerProfilerCommand type, const 
std::string params) override {
 
 Review comment:
   please pass std::string by const reference


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171447968
 
 

 ##
 File path: src/kvstore/kvstore_dist_server.h
 ##
 @@ -170,6 +187,33 @@ class KVStoreDistServer {
 app->Response(recved);
   }
 
+  void SetProfilerConfig(std::string params_str) {
+std::vector elems;
+mxnet::kvstore::split(params_str, ',', std::back_inserter(elems));
+std::vector ckeys;
+std::vector cvals;
+ckeys.reserve(elems.size());
+cvals.reserve(elems.size());
+
+for (int i=0; i < elems.size(); i++) {
+  std::vector parts;
+  mxnet::kvstore::split(elems[i], ':', std::back_inserter(parts));
+  CHECK_NOTNULL(parts[0].c_str());
+  CHECK_NOTNULL(parts[1].c_str());
+  if (parts[0] == "filename") {
+parts[1] = "rank" + std::to_string(ps::MyRank()) + "_" + parts[1];
+  }
+  char * ckey = new char[parts[0].length() + 1];
+  std::sprintf(ckey, "%s", parts[0].c_str());
+  ckeys.push_back(ckey);
+
+  char* cval = new char[parts[1].length() + 1];
 
 Review comment:
   nit: consistent  * placement 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171447768
 
 

 ##
 File path: src/kvstore/kvstore_dist_server.h
 ##
 @@ -159,6 +163,19 @@ class KVStoreDistServer {
   sync_mode_ = true;
 } else if (recved_type == CommandType::kSetGradientCompression) {
   gradient_compression_->DecodeParams(recved.body);
+} else if (recved_type == CommandType::kSetProfilerParams) {
+  // last char is the type of profiler command
+  KVStoreServerProfilerCommand profiler_command_type =
+  static_cast(recved.body.back() - '0');
+  if (profiler_command_type == KVStoreServerProfilerCommand::kSetConfig) {
 
 Review comment:
   prefer switch to many ifs


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171446913
 
 

 ##
 File path: include/mxnet/kvstore.h
 ##
 @@ -38,6 +38,11 @@
 #endif  // MXNET_USE_DIST_KVSTORE
 
 namespace mxnet {
+
+enum class KVStoreServerProfilerCommand {
 
 Review comment:
   nit: doc comment


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171447072
 
 

 ##
 File path: include/mxnet/c_api.h
 ##
 @@ -2005,6 +2005,16 @@ MXNET_DLL int 
MXKVStoreSendCommmandToServers(KVStoreHandle handle,
  int cmd_id,
  const char* cmd_body);
 
+
+MXNET_DLL int MXKVStoreSetServerProfilerConfig(KVStoreHandle handle,
 
 Review comment:
   documentation in comments, please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171446726
 
 

 ##
 File path: src/kvstore/kvstore_dist_server.h
 ##
 @@ -202,12 +203,12 @@ class KVStoreDistServer {
   if (parts[0] == "filename") {
 parts[1] = "rank" + std::to_string(ps::MyRank()) + "_" + parts[1];
   }
-  char * ckey = new char [parts[0].length()+1];
-  std::strcpy (ckey, parts[0].c_str());
+  char * ckey = new char[parts[0].length() + 1];
+  std::sprintf(ckey, "%s", parts[0].c_str());
 
 Review comment:
   why? this would be slower 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

cjolivier01 commented on a change in pull request #9933: Adding support to 
profile kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#discussion_r171446942
 
 

 ##
 File path: include/mxnet/kvstore.h
 ##
 @@ -361,6 +366,11 @@ class KVStore {
*/
   virtual void SendCommandToServers(int cmd_id, const std::string& cmd_body) { 
}
 
+  virtual void SetServerProfilerCommand(KVStoreServerProfilerCommand type,
 
 Review comment:
   doc comment


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9932: Fixing couple of bugs in profiler

[GitBox]

cjolivier01 commented on a change in pull request #9932: Fixing couple of bugs 
in profiler
URL: https://github.com/apache/incubator-mxnet/pull/9932#discussion_r171445123
 
 

 ##
 File path: src/profiler/profiler.h
 ##
 @@ -131,8 +131,9 @@ struct ProfileStat {
   size_t process_id_ = current_process_id();
 
   /*! \brief id of thread which operation run on */
-  std::thread::id thread_id_ = std::this_thread::get_id();  // Not yet seen a
-// case where this 
isn't valid
+  size_t thread_id_ = std::hash{}(std::this_thread::get_id());
+  // thread getid returns a hex, hashing it to get a size_t
 
 Review comment:
   nit: usually comment goes above the code?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #9932: Fixing couple of bugs in profiler

[GitBox]

cjolivier01 commented on a change in pull request #9932: Fixing couple of bugs 
in profiler
URL: https://github.com/apache/incubator-mxnet/pull/9932#discussion_r171444921
 
 

 ##
 File path: src/profiler/profiler.h
 ##
 @@ -131,8 +131,9 @@ struct ProfileStat {
   size_t process_id_ = current_process_id();
 
   /*! \brief id of thread which operation run on */
-  std::thread::id thread_id_ = std::this_thread::get_id();  // Not yet seen a
-// case where this 
isn't valid
+  size_t thread_id_ = std::hash{}(std::this_thread::get_id());
 
 Review comment:
   why do it here instead of emit time?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #9930: Support single array input for metric

[GitBox]

piiswrong commented on issue #9930: Support single array input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930#issuecomment-369448330
 
 
   when labels and preds are NDArray instead of list NDArray, the for loop is 
superfluous


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sxjscience opened a new pull request #9934: [Gluon] Support variant sequence length in gluon.RecurrentCell

[GitBox]

sxjscience opened a new pull request #9934: [Gluon] Support variant sequence 
length in gluon.RecurrentCell 
URL: https://github.com/apache/incubator-mxnet/pull/9934
 
 
   ## Description ##
   When the input sequences have different lengths, the common approach is to 
pad them to the same length and feed the padded data into the recurrent neural 
network. To deal with this scenario, this PR adds a new `valid_length` option 
in `unroll`. `valid_length` refers to the real length of the sequences before 
padding. When the `valid_length` is given, the last valid state will be 
returned and the padded portion in the output will be masked to be zero. This 
feature is essential for implementing a NMT model.
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] valid_length option in unroll, tests
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 opened a new pull request #9933: Adding support to profile kvstore server during distributed training

[GitBox]

rahul003 opened a new pull request #9933: Adding support to profile kvstore 
server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933
 
 
   ## Description ##
   Allows profiling of kvstore server through the python kvstore API
   
   kv.set_server_profiler_config
   kv.set_server_profiler_state
   
   Uses kvstore dist's support for sending command to servers to communicate 
messages from worker to server regarding profiling.
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Changes in c_api and python kvstore to add functions to set profiler 
configs, state, etc
   - [ ] KVDist passes these commands to server
   - [ ] KVDistServer calls the profiler API using above commands
   - [ ] Added an example in common/fit.py to use for all 
example/image-classification/* scripts
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 opened a new pull request #9932: Fixing couple of bugs in profiler

[GitBox]

rahul003 opened a new pull request #9932: Fixing couple of bugs in profiler
URL: https://github.com/apache/incubator-mxnet/pull/9932
 
 
   ## Description ##
   1. Fixes issue where thread id can be hex, and the hex symbols are not 
recognized by chrome tracing tool. Using hash to convert it to an integer
   
   2. The json file has some missing commas because of the first_flag 
condition. The reason for this seems lost at this point. Removing it bring back 
the comma necessary for a valid json. Verified that it does generate valid 
jsons by training a network. 
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - See description
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] juliusshufan commented on issue #9862: Fix a race condition in converting data layouts in MKLDNN.

[GitBox]

juliusshufan commented on issue #9862: Fix a race condition in converting data 
layouts in MKLDNN.
URL: https://github.com/apache/incubator-mxnet/pull/9862#issuecomment-369440792
 
 
   @zheng-da @cjolivier01 @piiswrong @pengzhao-intel  sorry for late response.
   The test result is positive, with 300 epoc, resnet50+cifar10
   top1 validation accuracy is 0.923478
   top1 training accuracy is 0.987500
   
   Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet-site] branch asf-site updated: Update README.md

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 4fa4b26  Update README.md
4fa4b26 is described below

commit 4fa4b266096aaf1d23db6bde98dc98646eb28bd1
Author: Sheng Zha 
AuthorDate: Wed Feb 28 17:10:44 2018 -0800

Update README.md
---
 README.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/README.md b/README.md
index 5a475f5..e5d9c30 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,5 @@
 # Apache MXNet Website
 
 [Development Notes](https://github.com/apache/incubator-mxnet/tree/master/docs)
+
+Note: This package is usually generated automatically, so manual changes to 
this package may be lost.

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] tornadomeet commented on issue #9880: TVM bridge support to JIT NDArray Function by TVM

[GitBox]

tornadomeet commented on issue #9880: TVM bridge support to JIT NDArray 
Function by TVM
URL: https://github.com/apache/incubator-mxnet/pull/9880#issuecomment-369440082
 
 
   @tqchen  yes, that fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn.

[GitBox]

marcoabreu commented on a change in pull request #9918: Update mkldnn to the 
newest & Add clang build test with mkldnn.
URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171437707
 
 

 ##
 File path: tests/ci_build/Dockerfile.cpu_clang
 ##
 @@ -19,3 +19,9 @@ RUN wget -O - http://apt.llvm.org/llvm-snapshot.gpg.key | 
apt-key add - && \
 apt-get install -y clang-3.9 clang-5.0 && \
 clang-3.9 --version && \
 clang-5.0 --version
+
+# Add MKLML library, compatiable with Ubuntu16.04
+RUN wget --no-check-certificate -O /tmp/mklml.tgz 
https://github.com/01org/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz
 
 Review comment:
   Please update all dockerfiles in that directory in order to use the correct 
mkl version


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] hetong007 commented on a change in pull request #9930: Support single array input for metric

[GitBox]

hetong007 commented on a change in pull request #9930: Support single array 
input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930#discussion_r171436622
 
 

 ##
 File path: python/mxnet/metric.py
 ##
 @@ -793,6 +793,8 @@ def update(self, labels, preds):
 
 if len(label.shape) == 1:
 label = label.reshape(label.shape[0], 1)
+if len(pred.shape) == 1:
+pred = pred.reshape(pred.shape[0], 1)
 
 
 Review comment:
   @piiswrong can we have your input?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhanghang1989 opened a new pull request #9931: Add axes support to Dropout for variational dropout in NLP

[GitBox]

zhanghang1989 opened a new pull request #9931: Add axes support to Dropout for 
variational dropout in NLP
URL: https://github.com/apache/incubator-mxnet/pull/9931
 
 
   add axes support to dropout for variational dropout  in NLP
   1. test pending
   2. MKL part hasn't been updated
   
   @szha Could you test this implementation? Could you also ping Yizhi for MKL 
implementation
   
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9928: host doc on s3

[GitBox]

marcoabreu commented on a change in pull request #9928: host doc on s3
URL: https://github.com/apache/incubator-mxnet/pull/9928#discussion_r171434356
 
 

 ##
 File path: tests/ci_build/deploy/ci_deploy_doc.sh
 ##
 @@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# Execute command within a docker container
 
 Review comment:
   "outside a docker container". AWS CLI is not available in docker.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #54: v1.1.0 manual update

[GitBox]

marcoabreu commented on issue #54: v1.1.0 manual update
URL: 
https://github.com/apache/incubator-mxnet-site/pull/54#issuecomment-369435492
 
 
   Please update the tag.txt file


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: organized installations (#9718)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 632fbc8  organized installations (#9718)
632fbc8 is described below

commit 632fbc8cfcc701dea596b03c6944f527d347f315
Author: Aaron Markham 
AuthorDate: Wed Feb 28 16:35:01 2018 -0800

organized installations (#9718)
---
 docs/install/index.md | 668 --
 1 file changed, 324 insertions(+), 344 deletions(-)

diff --git a/docs/install/index.md b/docs/install/index.md
index 8caedfb..ae8cc69 100644
--- a/docs/install/index.md
+++ b/docs/install/index.md
@@ -62,11 +62,46 @@ Indicate your preferred configuration. Then, follow the 
customized commands to i
 
 
 
-  
-
+
+
 
 The following installation instructions have been tested on Ubuntu 14.04 and 
16.04.
 
+
+
+
+
+**Step 1**  Install prerequisites - wget and latest pip.
+
+Installing *MXNet* with pip requires a latest version of `pip`. Install the 
latest version of `pip` by issuing the following command in the terminal.
+
+```bash
+$ sudo apt-get update
+$ sudo apt-get install -y wget python gcc
+$ wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
+```
+
+**Step 2** Install MXNet with OpenBLAS acceleration.
+
+```bash
+$ pip install mxnet
+```
+
+**Step 3**  Install [Graphviz](http://www.graphviz.org/). (Optional, needed 
for graph visualization using `mxnet.viz` package).
+```bash
+sudo apt-get install graphviz
+pip install graphviz
+```
+
+**Step 4**  Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
+
+**Experimental Choice** If You would like to install mxnet with Intel MKL, try 
the experimental pip package with MKL:
+```bash
+$ pip install mxnet-mkl
+```
+
+ 
+
 
 
 
@@ -121,41 +156,8 @@ pip install graphviz
 
 **Note**  You can read more about virtualenv 
[here](https://virtualenv.pypa.io/en/stable/userguide/).
 
-
-
-
-
-
-**Step 1**  Install prerequisites - wget and latest pip.
-
-Installing *MXNet* with pip requires a latest version of `pip`. Install the 
latest version of `pip` by issuing the following command in the terminal.
-
-```bash
-$ sudo apt-get update
-$ sudo apt-get install -y wget python gcc
-$ wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
-```
-
-**Step 2** Install MXNet with OpenBLAS acceleration.
-
-```bash
-$ pip install mxnet
-```
-
-**Step 3**  Install [Graphviz](http://www.graphviz.org/). (Optional, needed 
for graph visualization using `mxnet.viz` package).
-```bash
-sudo apt-get install graphviz
-pip install graphviz
-```
-
-**Step 4**  Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
-
-**Experimental Choice** If You would like to install mxnet with Intel MKL, try 
the experimental pip package with MKL:
-```bash
-$ pip install mxnet-mkl
-```
+ 
 
-
 
 
 
@@ -172,7 +174,7 @@ Follow the four steps in this [docker 
documentation](https://docs.docker.com/eng
 
 If you skip this step, you need to use *sudo* each time you invoke Docker.
 
-**Step 2** Pull the MXNet docker image.
+**Step 3** Pull the MXNet docker image.
 
 ```bash
 $ docker pull mxnet/python # Use sudo if you skip Step 2
@@ -187,9 +189,10 @@ REPOSITORY  TAG IMAGE ID   
 CREATED
 mxnet/pythonlatest  00d026968b3c3 weeks ago
 1.41 GB
 ```
 
-**Step 3** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
+**Step 4** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
+
+ 
 
-
 
 
 
@@ -264,17 +267,11 @@ pip install graphviz
 
 **Step 4** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
 
-
-
-
-
-
-
+
+
 
 
 
-
-
 
 
 The following installation instructions have been tested on Ubuntu 14.04 and 
16.04.
@@ -323,7 +320,7 @@ pip install graphviz
 $ pip install mxnet-cu90mkl
 ```
 
-
+ 
 
 
 
@@ -380,7 +377,7 @@ pip install graphviz
 
 **Note**  You can read more about virtualenv 
[here](https://virtualenv.pypa.io/en/stable/userguide/).
 
-
+ 
 
 
 
@@ -417,7 +414,7 @@ mxnet/pythongpu 493b2683c269
3 weeks ago
 
 **Step 5** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
 
-
+ 
 
 
 
@@ -491,87 +488,162 @@ pip install graphviz
 ```
 
 **Step 4** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
-
 
-
-
-
-
+ 
+ 
+ 
 
-
 
-
-  
-
+
+
 
-The following installation instructions have been tested on OSX Sierra and El 
Capitan.
+
+
+Building *MXNet* from source is a 2 step process.
+1. Build the *MXNet* core shared library, `libmxnet.so`, 

[GitHub] szha closed pull request #9718: organized installation instructions to fix render issue

[GitBox]

szha closed pull request #9718: organized installation instructions to fix 
render issue
URL: https://github.com/apache/incubator-mxnet/pull/9718
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/install/index.md b/docs/install/index.md
index 8caedfb7011..ae8cc698b21 100644
--- a/docs/install/index.md
+++ b/docs/install/index.md
@@ -62,11 +62,46 @@ Indicate your preferred configuration. Then, follow the 
customized commands to i
 
 
 
-  
-
+
+
 
 The following installation instructions have been tested on Ubuntu 14.04 and 
16.04.
 
+
+
+
+
+**Step 1**  Install prerequisites - wget and latest pip.
+
+Installing *MXNet* with pip requires a latest version of `pip`. Install the 
latest version of `pip` by issuing the following command in the terminal.
+
+```bash
+$ sudo apt-get update
+$ sudo apt-get install -y wget python gcc
+$ wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
+```
+
+**Step 2** Install MXNet with OpenBLAS acceleration.
+
+```bash
+$ pip install mxnet
+```
+
+**Step 3**  Install [Graphviz](http://www.graphviz.org/). (Optional, needed 
for graph visualization using `mxnet.viz` package).
+```bash
+sudo apt-get install graphviz
+pip install graphviz
+```
+
+**Step 4**  Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
+
+**Experimental Choice** If You would like to install mxnet with Intel MKL, try 
the experimental pip package with MKL:
+```bash
+$ pip install mxnet-mkl
+```
+
+ 
+
 
 
 
@@ -121,41 +156,8 @@ pip install graphviz
 
 **Note**  You can read more about virtualenv 
[here](https://virtualenv.pypa.io/en/stable/userguide/).
 
-
-
-
-
-
-**Step 1**  Install prerequisites - wget and latest pip.
-
-Installing *MXNet* with pip requires a latest version of `pip`. Install the 
latest version of `pip` by issuing the following command in the terminal.
-
-```bash
-$ sudo apt-get update
-$ sudo apt-get install -y wget python gcc
-$ wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
-```
-
-**Step 2** Install MXNet with OpenBLAS acceleration.
-
-```bash
-$ pip install mxnet
-```
-
-**Step 3**  Install [Graphviz](http://www.graphviz.org/). (Optional, needed 
for graph visualization using `mxnet.viz` package).
-```bash
-sudo apt-get install graphviz
-pip install graphviz
-```
-
-**Step 4**  Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
-
-**Experimental Choice** If You would like to install mxnet with Intel MKL, try 
the experimental pip package with MKL:
-```bash
-$ pip install mxnet-mkl
-```
+ 
 
-
 
 
 
@@ -172,7 +174,7 @@ Follow the four steps in this [docker 
documentation](https://docs.docker.com/eng
 
 If you skip this step, you need to use *sudo* each time you invoke Docker.
 
-**Step 2** Pull the MXNet docker image.
+**Step 3** Pull the MXNet docker image.
 
 ```bash
 $ docker pull mxnet/python # Use sudo if you skip Step 2
@@ -187,9 +189,10 @@ REPOSITORY  TAG IMAGE ID   
 CREATED
 mxnet/pythonlatest  00d026968b3c3 weeks ago
 1.41 GB
 ```
 
-**Step 3** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
+**Step 4** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
+
+ 
 
-
 
 
 
@@ -264,17 +267,11 @@ pip install graphviz
 
 **Step 4** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
 
-
-
-
-
-
-
+
+
 
 
 
-
-
 
 
 The following installation instructions have been tested on Ubuntu 14.04 and 
16.04.
@@ -323,7 +320,7 @@ pip install graphviz
 $ pip install mxnet-cu90mkl
 ```
 
-
+ 
 
 
 
@@ -380,7 +377,7 @@ pip install graphviz
 
 **Note**  You can read more about virtualenv 
[here](https://virtualenv.pypa.io/en/stable/userguide/).
 
-
+ 
 
 
 
@@ -417,7 +414,7 @@ mxnet/pythongpu 493b2683c269
3 weeks ago
 
 **Step 5** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
 
-
+ 
 
 
 
@@ -491,87 +488,162 @@ pip install graphviz
 ```
 
 **Step 4** Validate the installation by running simple MXNet code described 
[here](#validate-mxnet-installation).
-
 
-
-
-
-
+ 
+ 
+ 
 
-
 
-
-  
-
+
+
 
-The following installation instructions have been tested on OSX Sierra and El 
Capitan.
+
+
+Building *MXNet* from source is a 2 step process.
+1. Build the *MXNet* core shared library, `libmxnet.so`, from the C++ sources.
+2. Build the language specific bindings.
+
+**Minimum Requirements**
+1. [GCC 4.8](https://gcc.gnu.org/gcc-4.8/) or later to compile C++ 11.
+2. [GNU 

[GitHub] szha commented on issue #9718: organized installation instructions to fix render issue

[GitBox]

szha commented on issue #9718: organized installation instructions to fix 
render issue
URL: https://github.com/apache/incubator-mxnet/pull/9718#issuecomment-369433173
 
 
   
http://mxnet-doc.s3-accelerate.dualstack.amazonaws.com/PR-9718/install/index.html
 looks fine to me


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed issue #8681: training on P100 gets slower and slower

[GitBox]

eric-haibin-lin closed issue #8681: training on P100 gets slower and slower
URL: https://github.com/apache/incubator-mxnet/issues/8681
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #8681: training on P100 gets slower and slower

[GitBox]

eric-haibin-lin commented on issue #8681: training on P100 gets slower and 
slower
URL: 
https://github.com/apache/incubator-mxnet/issues/8681#issuecomment-369431713
 
 
   Closing due to inactivity


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #9557: update_on_kvstore error setting with multiple machines

[GitBox]

rahul003 commented on issue #9557: update_on_kvstore error setting with 
multiple machines
URL: 
https://github.com/apache/incubator-mxnet/issues/9557#issuecomment-369429911
 
 
   It doesn't look like a bug per se. The update is still done after pulling 
from the kvstore. I'm not sure if there is a reason why the update is done here 
rather than on the kvstore 
   
   Regarding the second point, that is just the init. And update_on_kvstore is 
only false when kvstore is local. Maybe there is one pull that can be saved, 
need to look into that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sxjscience commented on a change in pull request #9930: Support single array input for metric

[GitBox]

sxjscience commented on a change in pull request #9930: Support single array 
input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930#discussion_r171429077
 
 

 ##
 File path: python/mxnet/metric.py
 ##
 @@ -793,6 +793,8 @@ def update(self, labels, preds):
 
 if len(label.shape) == 1:
 label = label.reshape(label.shape[0], 1)
+if len(pred.shape) == 1:
+pred = pred.reshape(pred.shape[0], 1)
 
 
 Review comment:
   Not sure. I think shape=1 is more suitable to be the default.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on issue #9862: Fix a race condition in converting data layouts in MKLDNN.

[GitBox]

cjolivier01 commented on issue #9862: Fix a race condition in converting data 
layouts in MKLDNN.
URL: https://github.com/apache/incubator-mxnet/pull/9862#issuecomment-369429369
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] hetong007 commented on a change in pull request #9930: Support single array input for metric

[GitBox]

hetong007 commented on a change in pull request #9930: Support single array 
input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930#discussion_r171428505
 
 

 ##
 File path: python/mxnet/metric.py
 ##
 @@ -793,6 +793,8 @@ def update(self, labels, preds):
 
 if len(label.shape) == 1:
 label = label.reshape(label.shape[0], 1)
+if len(pred.shape) == 1:
+pred = pred.reshape(pred.shape[0], 1)
 
 
 Review comment:
   Is there a reason to set `shape=0` by default?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham opened a new pull request #54: v1.1.0 manual update

[GitBox]

aaronmarkham opened a new pull request #54: v1.1.0 manual update
URL: https://github.com/apache/incubator-mxnet-site/pull/54
 
 
   ## Description ##
   This is a manual build of the site using:
   ./build_site_tag.sh "1.1.0 1.0.0 0.12.1 0.12.0 0.11.0 master" 1.1.0 
http://mxnet.incubator.apache.org/
   
   It also has the install page for master copied from 1.1.0 to get rid of the 
formatting problem.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] indhub closed pull request #9929: save profile outputs to different file for troubleshooting purposes

[GitBox]

indhub closed pull request #9929: save profile outputs to different file for 
troubleshooting purposes
URL: https://github.com/apache/incubator-mxnet/pull/9929
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/tests/python/unittest/test_profiler.py 
b/tests/python/unittest/test_profiler.py
index b3a9a700ef9..7654cd26382 100644
--- a/tests/python/unittest/test_profiler.py
+++ b/tests/python/unittest/test_profiler.py
@@ -22,8 +22,7 @@
 import time
 import os
 
-def enable_profiler(run=True, continuous_dump=False, aggregate_stats=False):
-profile_filename = 'test_profile.json'
+def enable_profiler(profile_filename, run=True, continuous_dump=False, 
aggregate_stats=False):
 profiler.set_config(profile_symbolic=True,
 profile_imperative=True,
 profile_memory=True,
@@ -42,7 +41,7 @@ def test_profiler():
 begin_profiling_iter = 2
 end_profiling_iter = 4
 
-enable_profiler(False, False)
+enable_profiler('test_profiler.json', False, False)
 
 A = mx.sym.Variable('A')
 B = mx.sym.Variable('B')
@@ -77,14 +76,14 @@ def test_profiler():
 
 
 def test_profile_create_domain():
-enable_profiler()
+enable_profiler('test_profile_create_domain.json')
 domain = profiler.Domain(name='PythonDomain')
 print("Domain created: {}".format(str(domain)))
 profiler.set_state('stop')
 
 
 def test_profile_create_domain_dept():
-profiler.profiler_set_config(mode='symbolic', filename='temp.json')
+profiler.profiler_set_config(mode='symbolic', 
filename='test_profile_create_domain_dept.json')
 profiler.set_state('run')
 domain = profiler.Domain(name='PythonDomain')
 print("Domain created: {}".format(str(domain)))
@@ -103,7 +102,7 @@ def doLog():
 logging.info(template.format(*objects))
 
 logging.basicConfig()
-enable_profiler()
+enable_profiler('test_profile_task.json')
 python_domain = profiler.Domain('PythonDomain::test_profile_task')
 task = profiler.Task(python_domain, "test_profile_task")
 task.start()
@@ -129,7 +128,7 @@ def doLog():
 logging.info(template.format(*objects))
 
 logging.basicConfig()
-enable_profiler()
+enable_profiler('test_profile_frame.json')
 python_domain = profiler.Domain('PythonDomain::test_profile_frame')
 frame = profiler.Frame(python_domain, "test_profile_frame")
 frame.start()
@@ -156,7 +155,7 @@ def doLog():
 
 logging.basicConfig()
 if do_enable_profiler is True:
-  enable_profiler()
+enable_profiler('test_profile_event.json')
 event = profiler.Event("test_profile_event")
 event.start()
 start = time.time()
@@ -171,7 +170,7 @@ def doLog():
 
 
 def test_profile_tune_pause_resume():
-enable_profiler()
+enable_profiler('test_profile_tune_pause_resume.json')
 profiler.pause()
 # "test_profile_task" should *not* show up in tuning analysis
 test_profile_task()
@@ -199,7 +198,7 @@ def doLog(counter):
 logging.info(template.format(*objects))
 
 if do_enable_profiler is True:
-  enable_profiler()
+enable_profiler('test_profile_counter.json')
 python_domain = profiler.Domain('PythonDomain::test_profile_counter')
 counter = profiler.Counter(python_domain, 
"PythonCounter::test_profile_counter")
 counter.set_value(5)
@@ -213,7 +212,8 @@ def doLog(counter):
 
 
 def test_continuous_profile_and_instant_marker():
-enable_profiler(True, True, True)
+file_name = 'test_continuous_profile_and_instant_marker.json'
+enable_profiler(file_name, True, True, True)
 python_domain = profiler.Domain('PythonDomain::test_continuous_profile')
 last_file_size = 0
 for i in range(5):
@@ -223,7 +223,7 @@ def test_continuous_profile_and_instant_marker():
 test_profile_counter(False)
 profiler.dump(False)
 # File size should keep increasing
-new_file_size = os.path.getsize("test_profile.json")
+new_file_size = os.path.getsize(file_name)
 assert new_file_size >= last_file_size
 last_file_size = new_file_size
 profiler.dump(False)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: save profile outputs to different file for troubleshooting purposes (#9929)

[indhub]

This is an automated email from the ASF dual-hosted git repository.

indhub pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new eaaf72d  save profile outputs to different file for troubleshooting 
purposes (#9929)
eaaf72d is described below

commit eaaf72d848794ef81a8fda394591730d3fc8845b
Author: Chris Olivier 
AuthorDate: Wed Feb 28 16:10:16 2018 -0800

save profile outputs to different file for troubleshooting purposes (#9929)
---
 tests/python/unittest/test_profiler.py | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/tests/python/unittest/test_profiler.py 
b/tests/python/unittest/test_profiler.py
index b3a9a70..7654cd2 100644
--- a/tests/python/unittest/test_profiler.py
+++ b/tests/python/unittest/test_profiler.py
@@ -22,8 +22,7 @@ from mxnet import profiler
 import time
 import os
 
-def enable_profiler(run=True, continuous_dump=False, aggregate_stats=False):
-profile_filename = 'test_profile.json'
+def enable_profiler(profile_filename, run=True, continuous_dump=False, 
aggregate_stats=False):
 profiler.set_config(profile_symbolic=True,
 profile_imperative=True,
 profile_memory=True,
@@ -42,7 +41,7 @@ def test_profiler():
 begin_profiling_iter = 2
 end_profiling_iter = 4
 
-enable_profiler(False, False)
+enable_profiler('test_profiler.json', False, False)
 
 A = mx.sym.Variable('A')
 B = mx.sym.Variable('B')
@@ -77,14 +76,14 @@ def test_profiler():
 
 
 def test_profile_create_domain():
-enable_profiler()
+enable_profiler('test_profile_create_domain.json')
 domain = profiler.Domain(name='PythonDomain')
 print("Domain created: {}".format(str(domain)))
 profiler.set_state('stop')
 
 
 def test_profile_create_domain_dept():
-profiler.profiler_set_config(mode='symbolic', filename='temp.json')
+profiler.profiler_set_config(mode='symbolic', 
filename='test_profile_create_domain_dept.json')
 profiler.set_state('run')
 domain = profiler.Domain(name='PythonDomain')
 print("Domain created: {}".format(str(domain)))
@@ -103,7 +102,7 @@ def test_profile_task():
 logging.info(template.format(*objects))
 
 logging.basicConfig()
-enable_profiler()
+enable_profiler('test_profile_task.json')
 python_domain = profiler.Domain('PythonDomain::test_profile_task')
 task = profiler.Task(python_domain, "test_profile_task")
 task.start()
@@ -129,7 +128,7 @@ def test_profile_frame():
 logging.info(template.format(*objects))
 
 logging.basicConfig()
-enable_profiler()
+enable_profiler('test_profile_frame.json')
 python_domain = profiler.Domain('PythonDomain::test_profile_frame')
 frame = profiler.Frame(python_domain, "test_profile_frame")
 frame.start()
@@ -156,7 +155,7 @@ def test_profile_event(do_enable_profiler=True):
 
 logging.basicConfig()
 if do_enable_profiler is True:
-  enable_profiler()
+enable_profiler('test_profile_event.json')
 event = profiler.Event("test_profile_event")
 event.start()
 start = time.time()
@@ -171,7 +170,7 @@ def test_profile_event(do_enable_profiler=True):
 
 
 def test_profile_tune_pause_resume():
-enable_profiler()
+enable_profiler('test_profile_tune_pause_resume.json')
 profiler.pause()
 # "test_profile_task" should *not* show up in tuning analysis
 test_profile_task()
@@ -199,7 +198,7 @@ def test_profile_counter(do_enable_profiler=True):
 logging.info(template.format(*objects))
 
 if do_enable_profiler is True:
-  enable_profiler()
+enable_profiler('test_profile_counter.json')
 python_domain = profiler.Domain('PythonDomain::test_profile_counter')
 counter = profiler.Counter(python_domain, 
"PythonCounter::test_profile_counter")
 counter.set_value(5)
@@ -213,7 +212,8 @@ def test_profile_counter(do_enable_profiler=True):
 
 
 def test_continuous_profile_and_instant_marker():
-enable_profiler(True, True, True)
+file_name = 'test_continuous_profile_and_instant_marker.json'
+enable_profiler(file_name, True, True, True)
 python_domain = profiler.Domain('PythonDomain::test_continuous_profile')
 last_file_size = 0
 for i in range(5):
@@ -223,7 +223,7 @@ def test_continuous_profile_and_instant_marker():
 test_profile_counter(False)
 profiler.dump(False)
 # File size should keep increasing
-new_file_size = os.path.getsize("test_profile.json")
+new_file_size = os.path.getsize(file_name)
 assert new_file_size >= last_file_size
 last_file_size = new_file_size
 profiler.dump(False)

-- 
To stop receiving notification emails like this one, please contact
ind...@apache.org.

[GitHub] sxjscience commented on a change in pull request #9930: Support single array input for metric

[GitBox]

sxjscience commented on a change in pull request #9930: Support single array 
input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930#discussion_r171426781
 
 

 ##
 File path: python/mxnet/metric.py
 ##
 @@ -793,6 +793,8 @@ def update(self, labels, preds):
 
 if len(label.shape) == 1:
 label = label.reshape(label.shape[0], 1)
+if len(pred.shape) == 1:
+pred = pred.reshape(pred.shape[0], 1)
 
 
 Review comment:
   I think we can also revise the `check_label_shapes` above to be 
`check_label_shapes(labels, preds, 1)` to force the labels and preds have the 
same shape. Currently the default behavior is to just match the `ndim`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sxjscience commented on a change in pull request #9930: Support single array input for metric

[GitBox]

sxjscience commented on a change in pull request #9930: Support single array 
input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930#discussion_r171426781
 
 

 ##
 File path: python/mxnet/metric.py
 ##
 @@ -793,6 +793,8 @@ def update(self, labels, preds):
 
 if len(label.shape) == 1:
 label = label.reshape(label.shape[0], 1)
+if len(pred.shape) == 1:
+pred = pred.reshape(pred.shape[0], 1)
 
 
 Review comment:
   I think we can also revise the `check_label_shapes` above to be 
`check_label_shapes(labels, preds, True)` to force the labels and preds have 
the same shape.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9928: host doc on s3

[GitBox]

marcoabreu commented on a change in pull request #9928: host doc on s3
URL: https://github.com/apache/incubator-mxnet/pull/9928#discussion_r171426098
 
 

 ##
 File path: Jenkinsfile
 ##
 @@ -692,6 +692,7 @@ try {
 init_git()
 sh "make clean"
 sh "make docs"
+sh tests/ci_build/deploy/ci_deploy_doc.sh ${env.BRANCH_NAME} 
${env.CHANGE_ID}
 
 Review comment:
   Add quotation marks


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] hetong007 opened a new pull request #9930: Support single array input for metric

[GitBox]

hetong007 opened a new pull request #9930: Support single array input for metric
URL: https://github.com/apache/incubator-mxnet/pull/9930
 
 
   ## Description ##
   Fix #9865 by reshaping both `label` and `pred`.
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [X] Changes are complete (i.e. I finished coding on this PR)
   - [X] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ykim362 commented on issue #9906: Add CPU optimized docker which will be compiled with MKL-DNN

[GitBox]

ykim362 commented on issue #9906: Add CPU optimized docker which will be 
compiled with MKL-DNN
URL: https://github.com/apache/incubator-mxnet/pull/9906#issuecomment-369327083
 
 
   @marcoabreu , When I tested it, it worked fine for both cpu/mkl with python. 
If needed, we could update ubuntu version and etc. 
   
   In order to provide CPU optimized docker instances to the community, we'd 
like to create a new tag (mkl), here. 
https://hub.docker.com/r/mxnet/python/tags/ 
   The docker hub page links this docker directory - 
https://github.com/apache/incubator-mxnet/tree/master/docker .
   
   If there's any way to provide a cpu optimized tag, outside of this docker 
directory, could you please advise me so that I can add an instance?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kimjanik commented on issue #9906: Add CPU optimized docker which will be compiled with MKL-DNN

[GitBox]

kimjanik commented on issue #9906: Add CPU optimized docker which will be 
compiled with MKL-DNN
URL: https://github.com/apache/incubator-mxnet/pull/9906#issuecomment-369423878
 
 
   Actually, I am setting up the infrastructure to utilize these dockerfiles 
for CI and test coverage.  If the dockerfiles need updating, I think it would 
make more sense to get them up-to-date and make them be useful again.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #8679: is there monitor bug in mxnet code?

[GitBox]

anirudh2290 commented on issue #8679: is there monitor bug in mxnet code?
URL: 
https://github.com/apache/incubator-mxnet/issues/8679#issuecomment-369421367
 
 
   @liumilan i don't completely understand the question. Can you please expand 
on what is the problem you are trying to solve, and what is not working as 
expected.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #8681: training on P100 gets slower and slower

[GitBox]

anirudh2290 commented on issue #8681: training on P100 gets slower and slower
URL: 
https://github.com/apache/incubator-mxnet/issues/8681#issuecomment-369420902
 
 
   This issue seems abandoned. @eric-haibin-lin can you please help close this ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #9927: [Test] smaller success rate for test random

[GitBox]

piiswrong closed pull request #9927: [Test] smaller success rate for test random
URL: https://github.com/apache/incubator-mxnet/pull/9927
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/mxnet/test_utils.py b/python/mxnet/test_utils.py
index 1f3853b3cd8..91459de9d2a 100644
--- a/python/mxnet/test_utils.py
+++ b/python/mxnet/test_utils.py
@@ -1845,7 +1845,7 @@ def chi_square_check(generator, buckets, probs, 
nsamples=100):
 _, p = ss.chisquare(f_obs=obs_freq, f_exp=expected_freq)
 return p, obs_freq, expected_freq
 
-def verify_generator(generator, buckets, probs, nsamples=100, nrepeat=5, 
success_rate=0.25):
+def verify_generator(generator, buckets, probs, nsamples=100, nrepeat=5, 
success_rate=0.15):
 """Verify whether the generator is correct using chi-square testing.
 
 The test is repeated for "nrepeat" times and we check if the success rate 
is


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: set a lower success_rate (#9927)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new d87468e  set a lower success_rate (#9927)
d87468e is described below

commit d87468e563825e8f6d48cc8352b7db65e302ab2c
Author: Xingjian Shi 
AuthorDate: Wed Feb 28 15:19:35 2018 -0800

set a lower success_rate (#9927)
---
 python/mxnet/test_utils.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/python/mxnet/test_utils.py b/python/mxnet/test_utils.py
index 1f3853b..91459de 100644
--- a/python/mxnet/test_utils.py
+++ b/python/mxnet/test_utils.py
@@ -1845,7 +1845,7 @@ def chi_square_check(generator, buckets, probs, 
nsamples=100):
 _, p = ss.chisquare(f_obs=obs_freq, f_exp=expected_freq)
 return p, obs_freq, expected_freq
 
-def verify_generator(generator, buckets, probs, nsamples=100, nrepeat=5, 
success_rate=0.25):
+def verify_generator(generator, buckets, probs, nsamples=100, nrepeat=5, 
success_rate=0.15):
 """Verify whether the generator is correct using chi-square testing.
 
 The test is repeated for "nrepeat" times and we check if the success rate 
is

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[GitHub] piiswrong closed issue #9856: Flaky test_random.test_gamma_generator

[GitBox]

piiswrong closed issue #9856: Flaky test_random.test_gamma_generator
URL: https://github.com/apache/incubator-mxnet/issues/9856
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 opened a new pull request #9929: save profile outputs to different file for troubleshooting purposes

[GitBox]

cjolivier01 opened a new pull request #9929: save profile outputs to different 
file for troubleshooting purposes
URL: https://github.com/apache/incubator-mxnet/pull/9929
 
 
   ## Description ##
   save profile outputs to different file for troubleshooting purposes
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #8915: NVLink communication pattern updated

[GitBox]

rahul003 commented on issue #8915: NVLink communication pattern updated 
URL: https://github.com/apache/incubator-mxnet/pull/8915#issuecomment-360627429
 
 
   Could you please do these three things,
   1. add some comments on the flow of data for the reduce function? The flow 
is not easy to follow with code itself.
   2. ensure that tests/nightly/test_kvstore.py passes
   3. fix the extra residual issue 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #9921: [DISCUSSION] module.contrib.SparseModule API

[GitBox]

anirudh2290 commented on issue #9921: [DISCUSSION] module.contrib.SparseModule 
API
URL: 
https://github.com/apache/incubator-mxnet/issues/9921#issuecomment-369378869
 
 
   Why do we need both kvstore.row_sparse_pull and kvstore.pull in the example:
   
   ```
   for batch in batches:
   kvstore.row_sparse_pull(sparse_weight, useful_rows_for_this_batch) 
   kvstore.pull(dense_weights)
   mod.forward(batch)
   mod.backward(batch) 
   mod.update()
   # kvstore.push(grad) 
   ```
   
   Also, does update_on_kvstore=False not use the kvstore at all ? 
   
   For prepare, what if we have a static dictionary independent of batches ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] thinksanky commented on issue #9925: fixed the tag_list ordering to enable right ordered add link mapping

[GitBox]

thinksanky commented on issue #9925: fixed the tag_list ordering to enable 
right ordered add link mapping
URL: https://github.com/apache/incubator-mxnet/pull/9925#issuecomment-369388613
 
 
   Got the build working.
   updated the script to use 
   git submodule update --init --recursive


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #9928: host doc on s3

[GitBox]

szha commented on issue #9928: host doc on s3
URL: https://github.com/apache/incubator-mxnet/pull/9928#issuecomment-369386378
 
 
   Removed ACL update. Now public has read access (GetObject) for everything in 
the bucket. Slave has full read-write access to the objects in the bucket 
without ACL editing permission.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudhacharya commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

anirudhacharya commented on a change in pull request #9892: [WIP] Serde Module 
for Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171388136
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/__init__.py
 ##
 @@ -0,0 +1,26 @@
+# coding: utf-8
+"""import function"""
+import onnx
+from .import_onnx import GraphProto
+
+def import_model(model_file):
 
 Review comment:
   It will be used as follows - 
   ```python
   sym, params = 
mxnet.contrib.onnx._import.import_model("super_resolution.onnx")
   ```
   So onnx is already present in the package path. 
   Adding onnx in the method call could be redundant. It will get used like 
this 
   ```python
   sym, params = 
mxnet.contrib.onnx._import.import_onnx_model("super_resolution.onnx")
   ```
   I am not sure if we would want that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudhacharya commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

anirudhacharya commented on a change in pull request #9892: [WIP] Serde Module 
for Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171386740
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/backend_rep.py
 ##
 @@ -0,0 +1,70 @@
+# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 Review comment:
   I will update all the license and copyright statements.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudhacharya commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

anirudhacharya commented on a change in pull request #9892: [WIP] Serde Module 
for Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171386432
 
 

 ##
 File path: python/setup.py
 ##
 @@ -28,7 +28,7 @@
 else:
 from setuptools import setup
 from setuptools.extension import Extension
-kwargs = {'install_requires': ['numpy<=1.13.3,>=1.8.2', 
'requests==2.18.4', 'graphviz==0.8.1'], 'zip_safe': False}
+kwargs = {'install_requires': ['numpy<=1.13.3,>=1.8.2', 
'requests==2.18.4', 'graphviz==0.8.1', 'onnx>=1.0.1'], 'zip_safe': False}
 
 Review comment:
   will do.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #9928: host doc on s3

[GitBox]

szha commented on a change in pull request #9928: host doc on s3
URL: https://github.com/apache/incubator-mxnet/pull/9928#discussion_r171384424
 
 

 ##
 File path: tests/ci_build/deploy/ci_deploy_doc.sh
 ##
 @@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# Execute command within a docker container
+#
+# Usage: ci_deploy_doc.sh  
+#
+# PR_ID: the PR number
+#
+# BUILD_ID: the current build ID for the specified PR
+#
+aws s3 sync --delete docs/_build/html/ s3://mxnet-ci-doc/$1/$2 --acl 
public-read \
 
 Review comment:
   Yes, I will need to open up public-read for the bucket. The problem remains 
that changes to this script takes effect for anyone. The alternative is to put 
this command in Jenkinsfile, in which only committers' changes can take effect 
immediately.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #9928: host doc on s3

[GitBox]

marcoabreu commented on a change in pull request #9928: host doc on s3
URL: https://github.com/apache/incubator-mxnet/pull/9928#discussion_r171382625
 
 

 ##
 File path: tests/ci_build/deploy/ci_deploy_doc.sh
 ##
 @@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# Execute command within a docker container
+#
+# Usage: ci_deploy_doc.sh  
+#
+# PR_ID: the PR number
+#
+# BUILD_ID: the current build ID for the specified PR
+#
+aws s3 sync --delete docs/_build/html/ s3://mxnet-ci-doc/$1/$2 --acl 
public-read \
 
 Review comment:
   "acl public-read". Could a PR modify this flag in order to enable 
public-write? Could be problematic.
   
   Would it rather be possible to set permissions on a bucket-level that bubble 
down to every object?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171381419
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/backend_rep.py
 ##
 @@ -0,0 +1,70 @@
+# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 Review comment:
   2018, repeats in more files, please update.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171381281
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/backend.py
 ##
 @@ -0,0 +1,131 @@
+# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+# Licensed under the Apache License, Version 2.0 (the "License").
+# You may not use this file except in compliance with the License.
+# A copy of the License is located at
+# http://www.apache.org/licenses/LICENSE-2.0
+# or in the "license" file accompanying this file. This file is distributed
+# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+# express or implied. See the License for the specific language governing
+# permissions and limitations under the License.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,invalid-name
+"""backend wrapper for onnx test infrastructure"""
+from collections import namedtuple
 
 Review comment:
   nit: newline


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171381168
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/backend.py
 ##
 @@ -0,0 +1,131 @@
+# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 Review comment:
   2018


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171380850
 
 

 ##
 File path: python/mxnet/contrib/__init__.py
 ##
 @@ -28,3 +28,4 @@
 from . import tensorboard
 
 from . import text
+from . import serde
 
 Review comment:
   +1
   Serde can come in later as a facade on top of onnx and coreml


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171381032
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/__init__.py
 ##
 @@ -0,0 +1,26 @@
+# coding: utf-8
+"""import function"""
+import onnx
+from .import_onnx import GraphProto
+
+def import_model(model_file):
 
 Review comment:
   Propose to rename to import_onnx_model


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171381806
 
 

 ##
 File path: python/setup.py
 ##
 @@ -28,7 +28,7 @@
 else:
 from setuptools import setup
 from setuptools.extension import Extension
-kwargs = {'install_requires': ['numpy<=1.13.3,>=1.8.2', 
'requests==2.18.4', 'graphviz==0.8.1'], 'zip_safe': False}
+kwargs = {'install_requires': ['numpy<=1.13.3,>=1.8.2', 
'requests==2.18.4', 'graphviz==0.8.1', 'onnx>=1.0.1'], 'zip_safe': False}
 
 Review comment:
   Let's make ONNX a runtime dependency, in the future we can update it to be 
an install time dependency.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on a change in pull request #9892: [WIP] Serde Module for Import/Export of models between Onnx and Mxnet

[GitBox]

lupesko commented on a change in pull request #9892: [WIP] Serde Module for 
Import/Export of models between Onnx and Mxnet
URL: https://github.com/apache/incubator-mxnet/pull/9892#discussion_r171381201
 
 

 ##
 File path: python/mxnet/contrib/serde/_import/__init__.py
 ##
 @@ -0,0 +1,26 @@
+# coding: utf-8
 
 Review comment:
   Please add copyright headers


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #9921: [DISCUSSION] module.contrib.SparseModule API

[GitBox]

anirudh2290 commented on issue #9921: [DISCUSSION] module.contrib.SparseModule 
API
URL: 
https://github.com/apache/incubator-mxnet/issues/9921#issuecomment-369378869
 
 
   Why do we need both kvstore.row_sparse_pull and kvstore.pull in the example:
   
   ```
   for batch in batches:
   kvstore.row_sparse_pull(sparse_weight, useful_rows_for_this_batch) 
   kvstore.pull(dense_weights)
   mod.forward(batch)
   mod.backward(batch) 
   mod.update()
   # kvstore.push(grad) 
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha opened a new pull request #9928: host doc on s3

[GitBox]

szha opened a new pull request #9928: host doc on s3
URL: https://github.com/apache/incubator-mxnet/pull/9928
 
 
   ## Description ##
   host the doc built by CI for each PR on s3, as part of deploy step
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] ~All changes have test coverage:~ testing here in PR, as the change is 
in Jenkins
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] host doc for PR in s3


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #9893: Add constant parameter

[GitBox]

piiswrong closed pull request #9893: Add constant parameter
URL: https://github.com/apache/incubator-mxnet/pull/9893
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/api/python/gluon/gluon.md b/docs/api/python/gluon/gluon.md
index 2ae766fdcba..f523e649a45 100644
--- a/docs/api/python/gluon/gluon.md
+++ b/docs/api/python/gluon/gluon.md
@@ -34,6 +34,7 @@ in Python and then deploy with symbolic graph in C++ and 
Scala.
 :nosignatures:
 
 Parameter
+Constant
 ParameterDict
 ```
 
diff --git a/python/mxnet/gluon/parameter.py b/python/mxnet/gluon/parameter.py
index fa382859114..7dc72433926 100644
--- a/python/mxnet/gluon/parameter.py
+++ b/python/mxnet/gluon/parameter.py
@@ -18,8 +18,9 @@
 # coding: utf-8
 # pylint: disable=
 """Neural network parameter."""
-__all__ = ['DeferredInitializationError', 'Parameter', 'ParameterDict',
-   'tensor_types']
+__all__ = ['DeferredInitializationError', 'Parameter', 'Constant',
+   'ParameterDict', 'tensor_types']
+
 
 from collections import OrderedDict
 import warnings
@@ -459,6 +460,46 @@ def cast(self, dtype):
 autograd.mark_variables(self._data, self._grad, self.grad_req)
 
 
+class Constant(Parameter):
+"""A constant parameter for holding immutable tensors.
+`Constant`s are ignored by `autograd` and `Trainer`, thus their values
+will not change during training. But you can still update their values
+manually with the `set_data` method.
+
+`Constant`s can be created with either::
+
+const = mx.gluon.Constant('const', [[1,2],[3,4]])
+
+or::
+
+class Block(gluon.Block):
+def __init__(self, **kwargs):
+super(Block, self).__init__(**kwargs)
+self.const = self.params.get_constant('const', [[1,2],[3,4]])
+
+Parameter
+-
+name : str
+Name of the parameter.
+value : array-like
+Initial value for the constant.
+"""
+def __init__(self, name, value):
+if not isinstance(value, ndarray.NDArray):
+value = ndarray.array(value)
+self.value = value
+
+class Init(initializer.Initializer):
+def _init_weight(self, _, arr):
+value.copyto(arr)
+init_name = 'Constant_{}_{}'.format(name, id(self))
+initializer.alias(init_name)(Init)
+
+super(Constant, self).__init__(
+name, grad_req='null', shape=value.shape, dtype=value.dtype,
+init=init_name)
+
+
 class ParameterDict(object):
 """A dictionary managing a set of parameters.
 
@@ -548,6 +589,45 @@ def get(self, name, **kwargs):
 setattr(param, k, v)
 return param
 
+def get_constant(self, name, value=None):
+"""Retrieves a :py:class:`Constant` with name ``self.prefix+name``. If 
not found,
+:py:func:`get` will first try to retrieve it from "shared" dict. If 
still not
+found, :py:func:`get` will create a new :py:class:`Constant` with 
key-word
+arguments and insert it to self.
+
+Constants
+--
+name : str
+Name of the desired Constant. It will be prepended with this 
dictionary's
+prefix.
+value : array-like
+Initial value of constant.
+
+Returns
+---
+Constant
+The created or retrieved :py:class:`Constant`.
+"""
+name = self.prefix + name
+param = self._get_impl(name)
+if param is None:
+if value is None:
+raise KeyError("No constant named {}. Please specify value " \
+   "if you want to create a new constant.".format(
+   name))
+param = Constant(name, value)
+self._params[name] = param
+elif value is not None:
+assert isinstance(param, Constant), \
+"Parameter {} already exists but it is not a constant.".format(
+name)
+if isinstance(value, nd.NDArray):
+value = value.asnumpy()
+assert param.shape == value.shape and \
+(param.value.asnumpy() == value).all(), \
+"Constant {} already exists but it's value doesn't match new 
value"
+return param
+
 def update(self, other):
 """Copies all Parameters in ``other`` to self."""
 for k, v in other.items():
diff --git a/python/mxnet/registry.py b/python/mxnet/registry.py
index 4c131a1b755..0e4ac1c0b8c 100644
--- a/python/mxnet/registry.py
+++ b/python/mxnet/registry.py
@@ -69,7 +69,8 @@ def register(klass, name=None):
 assert issubclass(klass, base_class), \
 "Can only register 

[incubator-mxnet] branch master updated: Add constant parameter (#9893)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 8940fb3  Add constant parameter (#9893)
8940fb3 is described below

commit 8940fb3a40918ec5fc965377221517c4dc2a609b
Author: Eric Junyuan Xie 
AuthorDate: Wed Feb 28 12:40:47 2018 -0800

Add constant parameter (#9893)

* add constant parameter

* Update parameter.py

* Update parameter.py

* Update gluon.md
---
 docs/api/python/gluon/gluon.md  |  1 +
 python/mxnet/gluon/parameter.py | 84 -
 python/mxnet/registry.py|  3 +-
 tests/python/unittest/test_gluon.py | 28 +
 4 files changed, 113 insertions(+), 3 deletions(-)

diff --git a/docs/api/python/gluon/gluon.md b/docs/api/python/gluon/gluon.md
index 2ae766f..f523e64 100644
--- a/docs/api/python/gluon/gluon.md
+++ b/docs/api/python/gluon/gluon.md
@@ -34,6 +34,7 @@ in Python and then deploy with symbolic graph in C++ and 
Scala.
 :nosignatures:
 
 Parameter
+Constant
 ParameterDict
 ```
 
diff --git a/python/mxnet/gluon/parameter.py b/python/mxnet/gluon/parameter.py
index fa38285..7dc7243 100644
--- a/python/mxnet/gluon/parameter.py
+++ b/python/mxnet/gluon/parameter.py
@@ -18,8 +18,9 @@
 # coding: utf-8
 # pylint: disable=
 """Neural network parameter."""
-__all__ = ['DeferredInitializationError', 'Parameter', 'ParameterDict',
-   'tensor_types']
+__all__ = ['DeferredInitializationError', 'Parameter', 'Constant',
+   'ParameterDict', 'tensor_types']
+
 
 from collections import OrderedDict
 import warnings
@@ -459,6 +460,46 @@ class Parameter(object):
 autograd.mark_variables(self._data, self._grad, self.grad_req)
 
 
+class Constant(Parameter):
+"""A constant parameter for holding immutable tensors.
+`Constant`s are ignored by `autograd` and `Trainer`, thus their values
+will not change during training. But you can still update their values
+manually with the `set_data` method.
+
+`Constant`s can be created with either::
+
+const = mx.gluon.Constant('const', [[1,2],[3,4]])
+
+or::
+
+class Block(gluon.Block):
+def __init__(self, **kwargs):
+super(Block, self).__init__(**kwargs)
+self.const = self.params.get_constant('const', [[1,2],[3,4]])
+
+Parameter
+-
+name : str
+Name of the parameter.
+value : array-like
+Initial value for the constant.
+"""
+def __init__(self, name, value):
+if not isinstance(value, ndarray.NDArray):
+value = ndarray.array(value)
+self.value = value
+
+class Init(initializer.Initializer):
+def _init_weight(self, _, arr):
+value.copyto(arr)
+init_name = 'Constant_{}_{}'.format(name, id(self))
+initializer.alias(init_name)(Init)
+
+super(Constant, self).__init__(
+name, grad_req='null', shape=value.shape, dtype=value.dtype,
+init=init_name)
+
+
 class ParameterDict(object):
 """A dictionary managing a set of parameters.
 
@@ -548,6 +589,45 @@ class ParameterDict(object):
 setattr(param, k, v)
 return param
 
+def get_constant(self, name, value=None):
+"""Retrieves a :py:class:`Constant` with name ``self.prefix+name``. If 
not found,
+:py:func:`get` will first try to retrieve it from "shared" dict. If 
still not
+found, :py:func:`get` will create a new :py:class:`Constant` with 
key-word
+arguments and insert it to self.
+
+Constants
+--
+name : str
+Name of the desired Constant. It will be prepended with this 
dictionary's
+prefix.
+value : array-like
+Initial value of constant.
+
+Returns
+---
+Constant
+The created or retrieved :py:class:`Constant`.
+"""
+name = self.prefix + name
+param = self._get_impl(name)
+if param is None:
+if value is None:
+raise KeyError("No constant named {}. Please specify value " \
+   "if you want to create a new constant.".format(
+   name))
+param = Constant(name, value)
+self._params[name] = param
+elif value is not None:
+assert isinstance(param, Constant), \
+"Parameter {} already exists but it is not a constant.".format(
+name)
+if isinstance(value, nd.NDArray):
+value = value.asnumpy()
+assert param.shape == value.shape and \
+(param.value.asnumpy() == value).all(), \
+

[GitHub] anirudh2290 commented on issue #9813: Unable to save gluon model to symbolic network : neural style

[GitBox]

anirudh2290 commented on issue #9813: Unable to save gluon model to symbolic 
network : neural style
URL: 
https://github.com/apache/incubator-mxnet/issues/9813#issuecomment-369374611
 
 
   @samhodge this pr once merged should help your use case: #9893 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #9813: Unable to save gluon model to symbolic network : neural style

[GitBox]

anirudh2290 commented on issue #9813: Unable to save gluon model to symbolic 
network : neural style
URL: 
https://github.com/apache/incubator-mxnet/issues/9813#issuecomment-369374611
 
 
   @samhodge this pr once merged should help with your use case: #9893 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #9913: TODO list for Exception Handling Support

[GitBox]

anirudh2290 commented on issue #9913: TODO list for Exception Handling Support
URL: 
https://github.com/apache/incubator-mxnet/issues/9913#issuecomment-369372208
 
 
   Thanks @marcoabreu !


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sxjscience opened a new pull request #9927: [Test] smaller success rate for test random

[GitBox]

sxjscience opened a new pull request #9927: [Test] smaller success rate for 
test random
URL: https://github.com/apache/incubator-mxnet/pull/9927
 
 
   ## Description ##
   Set the success_rate to 15%. Should fix 
https://github.com/apache/incubator-mxnet/issues/9856 and 
https://github.com/apache/incubator-mxnet/issues/9843
   
   ## Checklist ##
   ### Essentials ###
   - [x] Passed code style checking (`make lint`)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] Set the success_rate to 15%
   
   ## Comments ##
   - [Randomness tests](https://en.wikipedia.org/wiki/Randomness_tests) are 
really difficult. There always exist specific seeds to break the test. This PR 
will not solve the problem, but will make the test fail less often.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #9923: Removed MXNET_USE_MKLDNN definition when option USE_MKLDNN is disabled

[GitBox]

marcoabreu commented on issue #9923: Removed MXNET_USE_MKLDNN definition when 
option USE_MKLDNN is disabled
URL: https://github.com/apache/incubator-mxnet/pull/9923#issuecomment-369370318
 
 
   @zheng-da could you verify?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pluskid commented on issue #8727: jenkins: julia build script

[GitBox]

pluskid commented on issue #8727: jenkins: julia build script
URL: https://github.com/apache/incubator-mxnet/pull/8727#issuecomment-369365031
 
 
   I think it is OK to keep the development in the main repo side: either way 
seems to have trade offs. I guess the important thing is to keep both sides in 
sync. Currently hard constraints is that the Julia package should be in a 
separate repo (being a mirror or not). 
   
   As for documents, sphinx might work but there might be some extra work to 
make the dynamically generated doc work. Also I see a much less motivation to 
merge and migrate the docs. The original setup should still work, right?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services