[GitHub] jaewoosong opened a new issue #7681: ValueError: Data provided by data_shapes don't match names specified by data_names ([] vs. ['data'])

[git]

jaewoosong opened a new issue #7681: ValueError: Data provided by data_shapes 
don't match names specified by data_names ([] vs. ['data'])
URL: https://github.com/apache/incubator-mxnet/issues/7681
 
 
   For bugs or installation issues, please provide the following information.
   The more information you provide, the more likely people will be able to 
help you.
   
   ## Environment info
   Operating System: Ubuntu 16.04
   
   Compiler: Python 3.6.1 (Anaconda 4.4.0, 64bit), installed by pip
   
   Package used (Python/R/Scala/Julia): Python
   
   MXNet version: 0.11.0
   
   Python version and distribution: Python 3.6.1 (Anaconda 4.4.0, 64bit), 
installed by pip
   
   ## Error Message:
   ```
   Traceback (most recent call last):
 File "/home/jaewoo/temp2.py", line 63, in 
   mod.fit(data_iter, num_epoch=5)
 File 
"/home/jaewoo/Programs/anaconda3/lib/python3.6/site-packages/mxnet/module/base_module.py",
 line 487, in fit
   self.forward_backward(data_batch)
 File 
"/home/jaewoo/Programs/anaconda3/lib/python3.6/site-packages/mxnet/module/base_module.py",
 line 191, in forward_backward
   self.forward(data_batch, is_train=True)
 File 
"/home/jaewoo/Programs/anaconda3/lib/python3.6/site-packages/mxnet/module/module.py",
 line 594, in forward
   self.reshape(new_dshape, new_lshape)
 File 
"/home/jaewoo/Programs/anaconda3/lib/python3.6/site-packages/mxnet/module/module.py",
 line 457, in reshape
   self.data_names, self.label_names, data_shapes, label_shapes)
 File 
"/home/jaewoo/Programs/anaconda3/lib/python3.6/site-packages/mxnet/module/base_module.py",
 line 71, in _parse_data_desc
   _check_names_match(data_names, data_shapes, 'data', True)
 File 
"/home/jaewoo/Programs/anaconda3/lib/python3.6/site-packages/mxnet/module/base_module.py",
 line 63, in _check_names_match
   raise ValueError(msg)
   ValueError: Data provided by data_shapes don't match names specified by 
data_names ([] vs. ['data'])
   ```
   
   ## Minimum reproducible example
   ```
   import mxnet as mx
   import os
   import subprocess
   import numpy as np
   import matplotlib.pyplot as plt
   import tarfile
   
   import warnings
   warnings.filterwarnings("ignore", category=DeprecationWarning)
   
   class SimpleIter(mx.io.DataIter):
   def __init__(self, data_names, data_shapes, data_gen,
label_names, label_shapes, label_gen, num_batches=10):
   self._provide_data = zip(data_names, data_shapes)
   self._provide_label = zip(label_names, label_shapes)
   self.num_batches = num_batches
   self.data_gen = data_gen
   self.label_gen = label_gen
   self.cur_batch = 0
   
   def __iter__(self):
   return self
   
   def reset(self):
   self.cur_batch = 0
   
   def __next__(self):
   return self.next()
   
   @property
   def provide_data(self):
   return self._provide_data
   
   @property
   def provide_label(self):
   return self._provide_label
   
   def next(self):
   if self.cur_batch < self.num_batches:
   self.cur_batch += 1
   data = [mx.nd.array(g(d[1])) for d,g in zip(self._provide_data, 
self.data_gen)]
   label = [mx.nd.array(g(d[1])) for d,g in 
zip(self._provide_label, self.label_gen)]
   return mx.io.DataBatch(data, label)
   else:
   raise StopIteration
   
   num_classes = 10
   net = mx.sym.Variable('data')
   net = mx.sym.FullyConnected(data=net, name='fc1', num_hidden=64)
   net = mx.sym.Activation(data=net, name='relu1', act_type="relu")
   net = mx.sym.FullyConnected(data=net, name='fc2', num_hidden=num_classes)
   net = mx.sym.SoftmaxOutput(data=net, name='softmax')
   print(net.list_arguments())
   print(net.list_outputs())
   
   n = 32
   data_iter = SimpleIter(['data'], [(n, 100)],
 [lambda s: np.random.uniform(-1, 1, s)],
 ['softmax_label'], [(n,)],
 [lambda s: np.random.randint(0, num_classes, s)])
   
   mod = mx.mod.Module(symbol=net)
   mod.fit(data_iter, num_epoch=5)
   ```
   Actually this code is same as the official mxnet tutorial. ( 
https://mxnet.incubator.apache.org/tutorials/basic/data.html )
   
   ## What have you tried to solve it?
   It seems `def _parse_data_desc(data_names, label_names, data_shapes, 
label_shapes)` and `def _check_names_match(data_names, data_shapes, name, 
throw)` have bugs.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7679: sign compare warnings

[git]

piiswrong closed pull request #7679: sign compare warnings
URL: https://github.com/apache/incubator-mxnet/pull/7679
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: sign compare warnings fixed (#7679)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 9cbcabc  sign compare warnings fixed (#7679)
9cbcabc is described below

commit 9cbcabcd4f7f8c70bc0f7f0b1dec6629fbba4733
Author: Rahul Huilgol 
AuthorDate: Wed Aug 30 22:19:59 2017 -0700

sign compare warnings fixed (#7679)
---
 src/operator/contrib/fft-inl.h   |  3 ++-
 src/operator/contrib/ifft-inl.h  |  3 ++-
 tests/cpp/operator/batchnorm_test.cc | 10 +-
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/src/operator/contrib/fft-inl.h b/src/operator/contrib/fft-inl.h
index 12474f1..afb22d1 100644
--- a/src/operator/contrib/fft-inl.h
+++ b/src/operator/contrib/fft-inl.h
@@ -208,7 +208,8 @@ class FFTOp : public Operator {
 
  private:
   FFTParam param_;
-  int dim_, stride_, num_compute, n_ffts;
+  int dim_, stride_, n_ffts;
+  size_t num_compute;
   bool init_cufft_;
 };  // class FFTOp
 #endif  // MXNET_USE_CUDA
diff --git a/src/operator/contrib/ifft-inl.h b/src/operator/contrib/ifft-inl.h
index 5e89c5b..99d2ed7 100644
--- a/src/operator/contrib/ifft-inl.h
+++ b/src/operator/contrib/ifft-inl.h
@@ -198,7 +198,8 @@ class IFFTOp : public Operator {
 
  private:
   IFFTParam param_;
-  int dim_, stride_, num_compute, n_iffts;
+  int dim_, stride_, n_iffts;
+  size_t num_compute;
   bool init_cufft_;
 };  // class IFFTOp
 
diff --git a/tests/cpp/operator/batchnorm_test.cc 
b/tests/cpp/operator/batchnorm_test.cc
index cd202ac..f64ba5b 100644
--- a/tests/cpp/operator/batchnorm_test.cc
+++ b/tests/cpp/operator/batchnorm_test.cc
@@ -1357,7 +1357,7 @@ TEST(BATCH_NORM, TestChannelAxisSaveAndLoad) {
 
 /*! \brief Insert the channel field `channelCount` into the shape at 
`channelAxis` position */
 static TShape MakeShape(const std::vector& shape,
-signed int channelAxis,
+unsigned int channelAxis,
 const size_t channelCount) {
   if (channelAxis < 0) {
 channelAxis += shape.size() + 1;
@@ -1369,7 +1369,7 @@ static TShape MakeShape(const std::vector& shape,
 newShape[x] = index_t(shape[x]);
   }
   newShape[channelAxis] = index_t(channelCount);
-  for (int x = channelAxis + 1; x < dim; ++x) {
+  for (index_t x = channelAxis + 1; x < dim; ++x) {
 newShape[x] = shape[x - 1];
   }
   return newShape;
@@ -1470,7 +1470,7 @@ static void runChannelAxisTest(
   ChannelAxisTestData::print("blob 2 output grad", 
info_c2.data_->c_.blob_out_grad_[0]);
 
   // Run both operators forward and backwards several times
-  for (int x = 0; x < numberOfPasses; ++x) {
+  for (index_t x = 0; x < numberOfPasses; ++x) {
 info_c1.data_->forward();
 info_c2.data_->forward();
 
@@ -1539,8 +1539,8 @@ TEST(BATCH_NORM, TestChannelAxis) {
   kwargs.push_back({"use_global_stats", tof[x2]});
   for (size_t x3 = 0; x3 < 2U; ++x3) {
 kwargs.push_back({"cudnn_off", tof[x3]});
-for (int g1 = 0; g1 < 2U; ++g1) {
-  for (int g2 = 0; g2 < 2U; ++g2) {
+for (index_t g1 = 0; g1 < 2U; ++g1) {
+  for (index_t g2 = 0; g2 < 2U; ++g2) {
 for (const std::vector  : shapes) {
   const int dim = static_cast(simpleShape.size());
   for (signed int channelAxis = -dim, shapeDim = dim;

-- 
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" '].

[GitHub] Godricly opened a new issue #7680: ndarray Slice with step

[git]

Godricly opened a new issue #7680: ndarray Slice with step
URL: https://github.com/apache/incubator-mxnet/issues/7680
 
 
   Current slicing still does not support slicing step. Is there any plan to 
support one? 
   ``` python
   a = mx.nd.zeros((10,10))
   b = a[:,::2]
   ```
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: [R] fix CI test (#7674)

[madjam]

This is an automated email from the ASF dual-hosted git repository.

madjam pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new ee5bf32  [R] fix CI test (#7674)
ee5bf32 is described below

commit ee5bf32a8eec0f2ce8e66463a89d6a48f591a032
Author: Qiang Kou (KK) 
AuthorDate: Wed Aug 30 22:42:05 2017 -0400

[R] fix CI test (#7674)
---
 R-package/tests/testthat/test_img_seg.R | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/R-package/tests/testthat/test_img_seg.R 
b/R-package/tests/testthat/test_img_seg.R
index fbca92e..b3400cd 100644
--- a/R-package/tests/testthat/test_img_seg.R
+++ b/R-package/tests/testthat/test_img_seg.R
@@ -90,7 +90,7 @@ context("Image segmentation")
 test_that("UNET", {
   list.of.packages <- c("imager")
   new.packages <- list.of.packages[!(list.of.packages %in% 
installed.packages()[,"Package"])]
-  if(length(new.packages)) install.packages(new.packages)
+  if(length(new.packages)) install.packages(new.packages, repos = 
"https://cloud.r-project.org/;)
   GetISBI_data()
   library(imager)
   IMG_SIZE <- 168
@@ -132,4 +132,4 @@ test_that("UNET", {
learning.rate = 0.05,
momentum = 0.99,
array.batch.size = 2)
-})
\ No newline at end of file
+})

-- 
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" '].

[GitHub] madjam closed pull request #7674: [R] fix CI test. close #7669

[git]

madjam closed pull request #7674: [R] fix CI test. close #7669
URL: https://github.com/apache/incubator-mxnet/pull/7674
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] astonzhang commented on a change in pull request #7678: Clean amalgamation ws and skip failing test

[git]

astonzhang commented on a change in pull request #7678: Clean amalgamation ws 
and skip failing test
URL: https://github.com/apache/incubator-mxnet/pull/7678#discussion_r136233692
 
 

 ##
 File path: tests/python/unittest/test_loss.py
 ##
 @@ -63,6 +63,9 @@ def get_net(num_hidden):
 
 
 def test_ce_loss():
+# Skipping this test to have nightly build passing.
+# There seems to be an issue and tracked at - issue 7677
+return
 
 Review comment:
   #7659 proposed some change in the unittest
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #7678: Clean amalgamation ws and skip failing test

[git]

szha commented on a change in pull request #7678: Clean amalgamation ws and 
skip failing test
URL: https://github.com/apache/incubator-mxnet/pull/7678#discussion_r136232563
 
 

 ##
 File path: tests/python/unittest/test_loss.py
 ##
 @@ -63,6 +63,9 @@ def get_net(num_hidden):
 
 
 def test_ce_loss():
+# Skipping this test to have nightly build passing.
+# There seems to be an issue and tracked at - issue 7677
+return
 
 Review comment:
   Drop the change on test_ce_loss test in this PR and work with @astonzhang to 
fix this test in his PR.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] thirdwing commented on issue #7669: [Jenkins CI] Broken R unit test : test_img_seg

[git]

thirdwing commented on issue #7669: [Jenkins CI] Broken R unit test : 
test_img_seg
URL: 
https://github.com/apache/incubator-mxnet/issues/7669#issuecomment-326169651
 
 
   CI passed now: 
https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/PR-7674/1/pipeline
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] madjam commented on a change in pull request #7678: Clean amalgamation ws and skip failing test

[git]

madjam commented on a change in pull request #7678: Clean amalgamation ws and 
skip failing test
URL: https://github.com/apache/incubator-mxnet/pull/7678#discussion_r136231251
 
 

 ##
 File path: tests/python/unittest/test_loss.py
 ##
 @@ -63,6 +63,9 @@ def get_net(num_hidden):
 
 
 def test_ce_loss():
+# Skipping this test to have nightly build passing.
+# There seems to be an issue and tracked at - issue 7677
+return
 
 Review comment:
   Roughly 15%. I do hear your concern but it doesn't look like there is a risk 
of this issue getting dropped. Do you have a better suggestion? Unreliable 
builds and flaky tests are hurting us in other ways.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #7421: Resolve more compile warnings

[git]

rahul003 commented on issue #7421: Resolve more compile warnings
URL: https://github.com/apache/incubator-mxnet/pull/7421#issuecomment-326166149
 
 
   Please don't merge this PR. I'm leaving this open for now, to hear back on 
the DType issue. 
   
   I've split the code changes in this to the new PR 7679. That can be merged 
if you don't see any issues.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 opened a new pull request #7679: sign compare warnings fixed

[git]

rahul003 opened a new pull request #7679: sign compare warnings fixed
URL: https://github.com/apache/incubator-mxnet/pull/7679
 
 
   These changes were introduced in 
https://github.com/apache/incubator-mxnet/pull/7421 with some other changes. 
   
   I'm splitting the PR for two reasons:
   1. These changes are simpler, can be merged while we wait on more 
information regarding the template
   2. That PR is now old, and master has changed a lot.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7656: [WIP] CSRNDArray Tutorial

[git]

eric-haibin-lin commented on a change in pull request #7656: [WIP] CSRNDArray 
Tutorial
URL: https://github.com/apache/incubator-mxnet/pull/7656#discussion_r136228913
 
 

 ##
 File path: docs/tutorials/sparse/csr.md
 ##
 @@ -0,0 +1,258 @@
+# CSRNDArray - NDArray in Compressed Sparse Row Storage Format
+
+Many real world datasets deal with high dimensional sparse feature vectors. 
For instance,
+in a recommendation system, the number of categories and users is in the order 
of millions,
+while most users only made a few purchases, leading to feature vectors with 
high sparsity
+(i.e. most of the elements are zeros).
+
+Storing and manipulating such large sparse matrices in the default dense 
structure results
+in wasted memory and processing on the zeros.
+To take advantage of the sparse structure of the matrix, the ``CSRNDArray`` in 
MXNet
+stores the matrix in [compressed sparse 
row(CSR)](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29)
 format
+and uses specialized algorithms in operators.
+The format is designed for 2D matrices with a large number of columns,
+and each row is sparse(i.e. with only a few nonzeros).
+For matrices of high sparsity (e.g. ~1% non-zeros), the advantage of 
``CSRNDArray`` over
+the existing ``NDArray`` is that
+
+- memory consumption is reduced significantly
+- certain operations (e.g. matrix-vector multiplication) are much faster
+
+Meanwhile, ``CSRNDArray`` inherits competitive features from ``NDArray`` such 
as
+lazy evaluation and automatic parallelization, which are not available in the
+scientific computing python package [SciPy](https://www.scipy.org/).
+
+Apart from often queried attributes such as **ndarray.shape**, 
**ndarray.dtype** and **ndarray.context**,
+you?ll also want to query **ndarray.stype**: the storage type of the NDArray. 
For a usual dense NDArray,
+the value of stype is **"default"**. For an CSRNDArray, the value of stype is 
**"csr"**.
+
+## Prerequisites
+
+To complete this tutorial, we need:
+
+- MXNet. See the instructions for your operating system in [Setup and 
Installation](http://mxnet.io/get_started/install.html)
+- [Jupyter](http://jupyter.org/)
+```
+pip install jupyter
+```
+- Scipy - A section of this tutorial uses Scipy package in python. If you 
don't have Scipy,
+the example in that section will be ignored.
+- GPUs - A section of this tutorial uses GPUs. If you don't have GPUs on your
+machine, simply set the variable gpu_device (set in the GPUs section of this
+tutorial) to mx.cpu().
+
+## Compressed Sparse Row Format
+
+A CSRNDArray represents a 2D matrix as three separate 1D arrays: **data**,
+**indptr** and **indices**, where the column indices for
+row ``i`` are stored in ``indices[indptr[i]:indptr[i+1]]`` in ascending order,
+and their corresponding values are stored in ``data[indptr[i]:indptr[i+1]]``.
+
+For example, the CSR representation for matrix
+```
+[[7, 0, 8, 0]
+ [0, 0, 0, 0]
+ [0, 9, 0, 0]]
+```
+is:
+```
+[7, 8, 9]  # data
+[0, 2, 1]  # indices
+[0, 2, 2, 3]   # indptr
+```
+
+Note that in MXNet, the column indices for a given row are always sorted in 
ascending order,
+and duplicated column entries for the same row are not allowed.
+
+## Array Creation
+
+There are a few different ways to create a `CSRNDArray`.
+
+* We can create a CSRNDArray with data, indices and indptr by using the 
`csr_matrix` function:
+
+```python
+import mxnet as mx
+import numpy as np
+# create a CSRNDArray with python lists
+shape = (3, 4)
+data_list = [7, 8, 9]
+indptr_list = [0, 2, 2, 3]
+indices_list = [0, 2, 1]
+a = mx.nd.sparse.csr_matrix(data_list, indptr_list, indices_list, shape)
+# create a CSRNDArray with numpy arrays
+data_np = np.array([7, 8, 9])
+indptr_np = np.array([0, 2, 2, 3])
+indices_np = np.array([0, 2, 1])
+b = mx.nd.sparse.csr_matrix(data_np, indptr_np, indices_np, shape)
+{'a':a, 'b':b}
+```
+
+* We can also create an MXNet CSRNDArray from a `scipy.sparse.csr.csr_matrix` 
object by using the `array` function:
+
+```python
+try:
+import scipy.sparse as spsp
+# generate a csr matrix in scipy
+c = spsp.csr.csr_matrix((data_np, indices_np, indptr_np), shape=shape)
+# create a CSRNDArray from a scipy csr object
+d = mx.nd.sparse.array(c)
+{'d':d}
+except ImportError:
+print("scipy package is required")
+```
+
+We can specify the element data type with the option `dtype`, which accepts a 
numpy
+type. By default, `float32` is used.
+
+```python
+# float32 is used in default
+e = mx.nd.sparse.array(a)
+# create a 16-bit float array
+f = mx.nd.array(a, dtype=np.float16)
+(e.dtype, f.dtype)
+```
+
+## Inspecting Arrays
+
+* We can inspect the contents of an `CSRNDArray` by filling
+its contents into a dense `numpy.ndarray` using the `asnumpy` function.
+
+```python
+a.asnumpy()
+```
+
+* We can also inspect the internal storage of a CSRNDArray by accessing 
attributes such as `indptr`, `indices` and 

[GitHub] eric-haibin-lin commented on a change in pull request #7656: [WIP] CSRNDArray Tutorial

[git]

eric-haibin-lin commented on a change in pull request #7656: [WIP] CSRNDArray 
Tutorial
URL: https://github.com/apache/incubator-mxnet/pull/7656#discussion_r136228840
 
 

 ##
 File path: docs/tutorials/sparse/csr.md
 ##
 @@ -0,0 +1,258 @@
+# CSRNDArray - NDArray in Compressed Sparse Row Storage Format
+
+Many real world datasets deal with high dimensional sparse feature vectors. 
For instance,
+in a recommendation system, the number of categories and users is in the order 
of millions,
+while most users only made a few purchases, leading to feature vectors with 
high sparsity
+(i.e. most of the elements are zeros).
+
+Storing and manipulating such large sparse matrices in the default dense 
structure results
+in wasted memory and processing on the zeros.
+To take advantage of the sparse structure of the matrix, the ``CSRNDArray`` in 
MXNet
+stores the matrix in [compressed sparse 
row(CSR)](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29)
 format
+and uses specialized algorithms in operators.
+The format is designed for 2D matrices with a large number of columns,
+and each row is sparse(i.e. with only a few nonzeros).
+For matrices of high sparsity (e.g. ~1% non-zeros), the advantage of 
``CSRNDArray`` over
+the existing ``NDArray`` is that
+
+- memory consumption is reduced significantly
+- certain operations (e.g. matrix-vector multiplication) are much faster
+
+Meanwhile, ``CSRNDArray`` inherits competitive features from ``NDArray`` such 
as
+lazy evaluation and automatic parallelization, which are not available in the
+scientific computing python package [SciPy](https://www.scipy.org/).
+
+Apart from often queried attributes such as **ndarray.shape**, 
**ndarray.dtype** and **ndarray.context**,
+you?ll also want to query **ndarray.stype**: the storage type of the NDArray. 
For a usual dense NDArray,
+the value of stype is **"default"**. For an CSRNDArray, the value of stype is 
**"csr"**.
+
+## Prerequisites
+
+To complete this tutorial, we need:
+
+- MXNet. See the instructions for your operating system in [Setup and 
Installation](http://mxnet.io/get_started/install.html)
+- [Jupyter](http://jupyter.org/)
+```
+pip install jupyter
+```
+- Scipy - A section of this tutorial uses Scipy package in python. If you 
don't have Scipy,
+the example in that section will be ignored.
+- GPUs - A section of this tutorial uses GPUs. If you don't have GPUs on your
+machine, simply set the variable gpu_device (set in the GPUs section of this
+tutorial) to mx.cpu().
+
+## Compressed Sparse Row Format
+
+A CSRNDArray represents a 2D matrix as three separate 1D arrays: **data**,
+**indptr** and **indices**, where the column indices for
+row ``i`` are stored in ``indices[indptr[i]:indptr[i+1]]`` in ascending order,
+and their corresponding values are stored in ``data[indptr[i]:indptr[i+1]]``.
+
+For example, the CSR representation for matrix
+```
+[[7, 0, 8, 0]
+ [0, 0, 0, 0]
+ [0, 9, 0, 0]]
+```
+is:
+```
+[7, 8, 9]  # data
+[0, 2, 1]  # indices
+[0, 2, 2, 3]   # indptr
+```
+
+Note that in MXNet, the column indices for a given row are always sorted in 
ascending order,
+and duplicated column entries for the same row are not allowed.
+
+## Array Creation
+
+There are a few different ways to create a `CSRNDArray`.
+
+* We can create a CSRNDArray with data, indices and indptr by using the 
`csr_matrix` function:
+
+```python
+import mxnet as mx
+import numpy as np
+# create a CSRNDArray with python lists
+shape = (3, 4)
+data_list = [7, 8, 9]
+indptr_list = [0, 2, 2, 3]
+indices_list = [0, 2, 1]
+a = mx.nd.sparse.csr_matrix(data_list, indptr_list, indices_list, shape)
+# create a CSRNDArray with numpy arrays
+data_np = np.array([7, 8, 9])
+indptr_np = np.array([0, 2, 2, 3])
+indices_np = np.array([0, 2, 1])
+b = mx.nd.sparse.csr_matrix(data_np, indptr_np, indices_np, shape)
+{'a':a, 'b':b}
+```
+
+* We can also create an MXNet CSRNDArray from a `scipy.sparse.csr.csr_matrix` 
object by using the `array` function:
+
+```python
+try:
+import scipy.sparse as spsp
+# generate a csr matrix in scipy
+c = spsp.csr.csr_matrix((data_np, indices_np, indptr_np), shape=shape)
+# create a CSRNDArray from a scipy csr object
+d = mx.nd.sparse.array(c)
 
 Review comment:
   Fixed now
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7656: [WIP] CSRNDArray Tutorial

[git]

eric-haibin-lin commented on a change in pull request #7656: [WIP] CSRNDArray 
Tutorial
URL: https://github.com/apache/incubator-mxnet/pull/7656#discussion_r136228844
 
 

 ##
 File path: docs/tutorials/sparse/csr.md
 ##
 @@ -0,0 +1,258 @@
+# CSRNDArray - NDArray in Compressed Sparse Row Storage Format
+
+Many real world datasets deal with high dimensional sparse feature vectors. 
For instance,
+in a recommendation system, the number of categories and users is in the order 
of millions,
+while most users only made a few purchases, leading to feature vectors with 
high sparsity
+(i.e. most of the elements are zeros).
+
+Storing and manipulating such large sparse matrices in the default dense 
structure results
+in wasted memory and processing on the zeros.
+To take advantage of the sparse structure of the matrix, the ``CSRNDArray`` in 
MXNet
+stores the matrix in [compressed sparse 
row(CSR)](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29)
 format
+and uses specialized algorithms in operators.
+The format is designed for 2D matrices with a large number of columns,
+and each row is sparse(i.e. with only a few nonzeros).
+For matrices of high sparsity (e.g. ~1% non-zeros), the advantage of 
``CSRNDArray`` over
+the existing ``NDArray`` is that
+
+- memory consumption is reduced significantly
+- certain operations (e.g. matrix-vector multiplication) are much faster
+
+Meanwhile, ``CSRNDArray`` inherits competitive features from ``NDArray`` such 
as
+lazy evaluation and automatic parallelization, which are not available in the
+scientific computing python package [SciPy](https://www.scipy.org/).
+
+Apart from often queried attributes such as **ndarray.shape**, 
**ndarray.dtype** and **ndarray.context**,
+you?ll also want to query **ndarray.stype**: the storage type of the NDArray. 
For a usual dense NDArray,
+the value of stype is **"default"**. For an CSRNDArray, the value of stype is 
**"csr"**.
+
+## Prerequisites
+
+To complete this tutorial, we need:
+
+- MXNet. See the instructions for your operating system in [Setup and 
Installation](http://mxnet.io/get_started/install.html)
+- [Jupyter](http://jupyter.org/)
+```
+pip install jupyter
+```
+- Scipy - A section of this tutorial uses Scipy package in python. If you 
don't have Scipy,
+the example in that section will be ignored.
+- GPUs - A section of this tutorial uses GPUs. If you don't have GPUs on your
+machine, simply set the variable gpu_device (set in the GPUs section of this
+tutorial) to mx.cpu().
+
+## Compressed Sparse Row Format
+
+A CSRNDArray represents a 2D matrix as three separate 1D arrays: **data**,
+**indptr** and **indices**, where the column indices for
+row ``i`` are stored in ``indices[indptr[i]:indptr[i+1]]`` in ascending order,
+and their corresponding values are stored in ``data[indptr[i]:indptr[i+1]]``.
+
+For example, the CSR representation for matrix
+```
+[[7, 0, 8, 0]
+ [0, 0, 0, 0]
+ [0, 9, 0, 0]]
+```
+is:
+```
+[7, 8, 9]  # data
+[0, 2, 1]  # indices
+[0, 2, 2, 3]   # indptr
+```
+
+Note that in MXNet, the column indices for a given row are always sorted in 
ascending order,
+and duplicated column entries for the same row are not allowed.
+
+## Array Creation
+
+There are a few different ways to create a `CSRNDArray`.
+
+* We can create a CSRNDArray with data, indices and indptr by using the 
`csr_matrix` function:
+
+```python
+import mxnet as mx
+import numpy as np
+# create a CSRNDArray with python lists
+shape = (3, 4)
+data_list = [7, 8, 9]
+indptr_list = [0, 2, 2, 3]
+indices_list = [0, 2, 1]
+a = mx.nd.sparse.csr_matrix(data_list, indptr_list, indices_list, shape)
+# create a CSRNDArray with numpy arrays
+data_np = np.array([7, 8, 9])
+indptr_np = np.array([0, 2, 2, 3])
+indices_np = np.array([0, 2, 1])
+b = mx.nd.sparse.csr_matrix(data_np, indptr_np, indices_np, shape)
+{'a':a, 'b':b}
+```
+
+* We can also create an MXNet CSRNDArray from a `scipy.sparse.csr.csr_matrix` 
object by using the `array` function:
+
+```python
+try:
+import scipy.sparse as spsp
+# generate a csr matrix in scipy
+c = spsp.csr.csr_matrix((data_np, indices_np, indptr_np), shape=shape)
+# create a CSRNDArray from a scipy csr object
+d = mx.nd.sparse.array(c)
+{'d':d}
+except ImportError:
 
 Review comment:
   Fixed now
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #7678: Clean amalgamation ws and skip failing test

[git]

szha commented on a change in pull request #7678: Clean amalgamation ws and 
skip failing test
URL: https://github.com/apache/incubator-mxnet/pull/7678#discussion_r136228780
 
 

 ##
 File path: tests/python/unittest/test_loss.py
 ##
 @@ -63,6 +63,9 @@ def get_net(num_hidden):
 
 
 def test_ce_loss():
+# Skipping this test to have nightly build passing.
+# There seems to be an issue and tracked at - issue 7677
+return
 
 Review comment:
   This is not how it should be done. Aston is fixing loss problems in #7659.
   
   How frequent is this test failing.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy opened a new pull request #7678: Clean amalgamation ws and skip failing test

[git]

sandeep-krishnamurthy opened a new pull request #7678: Clean amalgamation ws 
and skip failing test
URL: https://github.com/apache/incubator-mxnet/pull/7678
 
 
   @madjam 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #7677: Issue with Loss function

[git]

szha commented on issue #7677: Issue with Loss function
URL: 
https://github.com/apache/incubator-mxnet/issues/7677#issuecomment-326162663
 
 
   @astonzhang is already looking at this and will pr shortly.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy opened a new issue #7677: Issue with Loss function

[git]

sandeep-krishnamurthy opened a new issue #7677: Issue with Loss function
URL: https://github.com/apache/incubator-mxnet/issues/7677
 
 
   @szha @astonzhang @madjam 
   test_ce_loss is failing too frequently leading to failures in master CI 
builds. Based on the conversation, it looks like Aston is working on the fix. 
In the meantime, as part of stabilizing CI builds and avoid other issues 
getting in unnoticed, I am skipping this test.
   This issue should help to track in the fix and re-enabling the test_ce_loss.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on a change in pull request #7662: Isolated benchmarking support

[git]

anirudh2290 commented on a change in pull request #7662: Isolated benchmarking 
support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136225499
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -181,7 +220,7 @@ def row_sparse_pull(kv, key, data, slices, weight_array, 
priority):
 train_data = mx.io.LibSVMIter(data_libsvm=path, data_shape=(feature_dim,),
   batch_size=batch_size, num_parts=num_worker,
   part_index=rank)
-if dummy_iter:
+if dummy_iter or compute_only:
 
 Review comment:
   Nice catch. This was a mistake. Added communication_only.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on a change in pull request #7662: Isolated benchmarking support

[git]

anirudh2290 commented on a change in pull request #7662: Isolated benchmarking 
support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136225396
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -140,11 +166,24 @@ def row_sparse_pull(kv, key, data, slices, weight_array, 
priority):
 dummy_iter = args.dummy_iter
 dataset = args.dataset
 log_level = args.sparse_log_level
+io_only = args.io_only
+compute_only = args.compute_only
+communication_only = args.communication_only
+num_cores = multiprocessing.cpu_count()
+assert (compute_only + io_only + communication_only <= 1), "Only one of 
compute_only, io_only, communication_only can be set"
+if compute_only or io_only:
+assert not kvstore, "when compute_only or io_only is set, kvstore 
should be None"
+num_batch = datasets[dataset]['lc'] / batch_size if num_batch == 
 else num_batch
 
 Review comment:
   Changed.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on a change in pull request #7662: Isolated benchmarking support

[git]

anirudh2290 commented on a change in pull request #7662: Isolated benchmarking 
support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136225380
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -48,6 +49,12 @@
 help='whether to call update_metric')
 parser.add_argument('--enable-logging-for', default="0",
 help="Enable logging for the specified list of workers")
+parser.add_argument('--io-only', action='store_true',
 
 Review comment:
   Changed.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin opened a new pull request #7676: [WIP] Fix gpu test and nested try catch in Imperative Invoke

[git]

eric-haibin-lin opened a new pull request #7676: [WIP] Fix gpu test and nested 
try catch in Imperative Invoke 
URL: https://github.com/apache/incubator-mxnet/pull/7676
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alexandraj777 commented on a change in pull request #7673: Update docs/README link and folder name

[git]

alexandraj777 commented on a change in pull request #7673: Update docs/README 
link and folder name
URL: https://github.com/apache/incubator-mxnet/pull/7673#discussion_r136222738
 
 

 ##
 File path: docs/build_version_doc/build_all_version.sh
 ##
 @@ -24,7 +24,7 @@
 tag_list="0.11.0.rc3 master"
 
 mxnet_url="https://github.com/apache/incubator-mxnet.git;
-mxnet_folder="apache_mxnet"
+mxnet_folder="mxnet"
 
 Review comment:
   cc @piiswrong, I didn't know if this folder was purposely different from 
`mxnet` to avoid conflicts
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alexandraj777 commented on a change in pull request #7673: Update docs/README link and folder name

[git]

alexandraj777 commented on a change in pull request #7673: Update docs/README 
link and folder name
URL: https://github.com/apache/incubator-mxnet/pull/7673#discussion_r136221617
 
 

 ##
 File path: docs/README.md
 ##
 @@ -2,13 +2,13 @@
 
 A built version of document is available at http://mxnet.io
 
-To build the documents locally, we need to first install [docker](docker.com).
+To build the documents locally, we need to first install 
[docker](https://docker.com).
 Then use the following commands to clone and
 build the documents.
 
 ```bash
 git clone --recursive https://github.com/apache/incubator-mxnet.git
-cd mxnet && make docs
+cd incubator-mxnet && make docs
 
 Review comment:
   Yeah I'd be happy to
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy opened a new issue #7675: [Jenkins CI] Broken R Unit test during build

[git]

sandeep-krishnamurthy opened a new issue #7675: [Jenkins CI] Broken R Unit test 
during build
URL: https://github.com/apache/incubator-mxnet/issues/7675
 
 
   @thirdwing 
   
   R GPU unit tests are failing more frequently on the master branch. Can you 
please take a look at this?
   Example failed CI build - 
https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/master/272/pipeline/
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #7673: Update docs/README link and folder name

[git]

piiswrong commented on a change in pull request #7673: Update docs/README link 
and folder name
URL: https://github.com/apache/incubator-mxnet/pull/7673#discussion_r136219668
 
 

 ##
 File path: docs/README.md
 ##
 @@ -2,13 +2,13 @@
 
 A built version of document is available at http://mxnet.io
 
-To build the documents locally, we need to first install [docker](docker.com).
+To build the documents locally, we need to first install 
[docker](https://docker.com).
 Then use the following commands to clone and
 build the documents.
 
 ```bash
 git clone --recursive https://github.com/apache/incubator-mxnet.git
-cd mxnet && make docs
+cd incubator-mxnet && make docs
 
 Review comment:
   This is all the places where this is needed.
   ```
   c4b301d74d83:mxnet junyuanx$ git grep 
"https://github.com/apache/incubator-mxnet.git;
   docs/README.md:git clone --recursive 
https://github.com/apache/incubator-mxnet.git
   docs/build_version_doc/AddVersion.py:
'git clone --recursive https://github.com/apache/incubator-mxnet.git')
   docs/build_version_doc/AddVersion.py:
'git clone --recursive https://github.com/apache/incubator-mxnet.git '
   
docs/build_version_doc/build_all_version.sh:mxnet_url="https://github.com/apache/incubator-mxnet.git;
   ```
   Would you like to do the change at these 3 places?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] thirdwing opened a new pull request #7674: [R] fix CI test. close #7669

[git]

thirdwing opened a new pull request #7674: [R] fix CI test. close #7669
URL: https://github.com/apache/incubator-mxnet/pull/7674
 
 
   @sandeep-krishnamurthy
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7662: Isolated benchmarking support

[git]

eric-haibin-lin commented on a change in pull request #7662: Isolated 
benchmarking support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136216229
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -181,7 +220,7 @@ def row_sparse_pull(kv, key, data, slices, weight_array, 
priority):
 train_data = mx.io.LibSVMIter(data_libsvm=path, data_shape=(feature_dim,),
   batch_size=batch_size, num_parts=num_worker,
   part_index=rank)
-if dummy_iter:
+if dummy_iter or compute_only:
 
 Review comment:
   Is DummyIter not used for communication_only
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7662: Isolated benchmarking support

[git]

eric-haibin-lin commented on a change in pull request #7662: Isolated 
benchmarking support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136215726
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -140,11 +166,24 @@ def row_sparse_pull(kv, key, data, slices, weight_array, 
priority):
 dummy_iter = args.dummy_iter
 dataset = args.dataset
 log_level = args.sparse_log_level
+io_only = args.io_only
+compute_only = args.compute_only
+communication_only = args.communication_only
+num_cores = multiprocessing.cpu_count()
+assert (compute_only + io_only + communication_only <= 1), "Only one of 
compute_only, io_only, communication_only can be set"
+if compute_only or io_only:
+assert not kvstore, "when compute_only or io_only is set, kvstore 
should be None"
+num_batch = datasets[dataset]['lc'] / batch_size if num_batch == 
 else num_batch
 
 Review comment:
   replace   with macro 
   num_batch =  MAX_NUM_BATCH 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7662: Isolated benchmarking support

[git]

eric-haibin-lin commented on a change in pull request #7662: Isolated 
benchmarking support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136216109
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -48,6 +49,12 @@
 help='whether to call update_metric')
 parser.add_argument('--enable-logging-for', default="0",
 help="Enable logging for the specified list of workers")
+parser.add_argument('--io-only', action='store_true',
 
 Review comment:
   I actually prefer one option for `measure_only = IO/COMP/COMM, default to 
None`
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alexandraj777 commented on a change in pull request #7673: Update docs/README link and folder name

[git]

alexandraj777 commented on a change in pull request #7673: Update docs/README 
link and folder name
URL: https://github.com/apache/incubator-mxnet/pull/7673#discussion_r136212216
 
 

 ##
 File path: docs/README.md
 ##
 @@ -2,13 +2,13 @@
 
 A built version of document is available at http://mxnet.io
 
-To build the documents locally, we need to first install [docker](docker.com).
+To build the documents locally, we need to first install 
[docker](https://docker.com).
 Then use the following commands to clone and
 build the documents.
 
 ```bash
 git clone --recursive https://github.com/apache/incubator-mxnet.git
-cd mxnet && make docs
+cd incubator-mxnet && make docs
 
 Review comment:
   @piiswrong excellent suggestion! 
   
   Let me know if you've like me to remove this particular change so you can 
defer it to a larger discussion/pr.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #7673: Update docs/README link and folder name

[git]

piiswrong commented on a change in pull request #7673: Update docs/README link 
and folder name
URL: https://github.com/apache/incubator-mxnet/pull/7673#discussion_r136208922
 
 

 ##
 File path: docs/README.md
 ##
 @@ -2,13 +2,13 @@
 
 A built version of document is available at http://mxnet.io
 
-To build the documents locally, we need to first install [docker](docker.com).
+To build the documents locally, we need to first install 
[docker](https://docker.com).
 Then use the following commands to clone and
 build the documents.
 
 ```bash
 git clone --recursive https://github.com/apache/incubator-mxnet.git
-cd mxnet && make docs
+cd incubator-mxnet && make docs
 
 Review comment:
   This seems to be happening at a lot of places after we moved repo. Should we 
change to 
   `git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet` 
everywhere? @kevinthesun  
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7668: add -1 indexing to ndarray

[git]

piiswrong closed pull request #7668: add -1 indexing to ndarray
URL: https://github.com/apache/incubator-mxnet/pull/7668
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: add -1 indexing to ndarray (#7668)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new de4c6e4  add -1 indexing to ndarray (#7668)
de4c6e4 is described below

commit de4c6e401379a49bab46c68dbbf275991e682560
Author: Eric Junyuan Xie 
AuthorDate: Wed Aug 30 15:43:00 2017 -0700

add -1 indexing to ndarray (#7668)
---
 python/mxnet/ndarray/ndarray.py | 21 +++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/python/mxnet/ndarray/ndarray.py b/python/mxnet/ndarray/ndarray.py
index e497ea6..d5d4bb2 100644
--- a/python/mxnet/ndarray/ndarray.py
+++ b/python/mxnet/ndarray/ndarray.py
@@ -599,8 +599,25 @@ fixed-size items.
 array([], shape=(0, 2), dtype=float32)
 """
 handle = NDArrayHandle()
-start = mx_uint(start) if start else mx_uint(0)
-stop = mx_uint(stop) if stop else mx_uint(self.shape[0])
+if start is None:
+start = mx_uint(0)
+elif start < 0:
+length = self.shape[0]
+start += length
+assert start >= 0, "Slicing start %d exceeds limit of 
%d"%(start-length, length)
+start = mx_uint(start)
+else:
+start = mx_uint(start)
+if stop is None:
+stop = mx_uint(self.shape[0])
+elif stop < 0:
+length = self.shape[0]
+stop += length
+assert stop >= 0, "Slicing end %d exceeds limit of 
%d"%(stop-length, length)
+stop = mx_uint(stop)
+else:
+stop = mx_uint(stop)
+
 check_call(_LIB.MXNDArraySlice(
 self.handle, start, stop, ctypes.byref(handle)))
 return NDArray(handle=handle, writable=self.writable)

-- 
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" '].

[GitHub] madjam commented on issue #7672: Add missing license header

[git]

madjam commented on issue #7672: Add missing license header
URL: https://github.com/apache/incubator-mxnet/pull/7672#issuecomment-326134310
 
 
   Taken care of here: 
https://github.com/apache/incubator-mxnet/commit/42a43ba45d1248fa75941a6b82f21840c67e52f8
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] madjam closed pull request #7672: Add missing license header

[git]

madjam closed pull request #7672: Add missing license header
URL: https://github.com/apache/incubator-mxnet/pull/7672
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alexandraj777 opened a new pull request #7673: Update docs/README link and folder name

[git]

alexandraj777 opened a new pull request #7673: Update docs/README link and 
folder name
URL: https://github.com/apache/incubator-mxnet/pull/7673
 
 
   I noticed a few small typos in the README and the docs and wanted to help to 
fix them. I'm not sure how contributing changes to projects like this normally 
works, so please let me know if I should change anything to do with my 
branching, forking, etc. Thanks!
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] bhavinthaker commented on a change in pull request #7656: CSRNDArray Tutorial

[git]

bhavinthaker commented on a change in pull request #7656: CSRNDArray Tutorial
URL: https://github.com/apache/incubator-mxnet/pull/7656#discussion_r135953019
 
 

 ##
 File path: docs/tutorials/sparse/csr.md
 ##
 @@ -0,0 +1,258 @@
+# CSRNDArray - NDArray in Compressed Sparse Row Storage Format
+
+Many real world datasets deal with high dimensional sparse feature vectors. 
For instance,
+in a recommendation system, the number of categories and users is in the order 
of millions,
+while most users only made a few purchases, leading to feature vectors with 
high sparsity
+(i.e. most of the elements are zeros).
+
+Storing and manipulating such large sparse matrices in the default dense 
structure results
+in wasted memory and processing on the zeros.
+To take advantage of the sparse structure of the matrix, the ``CSRNDArray`` in 
MXNet
+stores the matrix in [compressed sparse 
row(CSR)](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29)
 format
+and uses specialized algorithms in operators.
+The format is designed for 2D matrices with a large number of columns,
+and each row is sparse(i.e. with only a few nonzeros).
+For matrices of high sparsity (e.g. ~1% non-zeros), the advantage of 
``CSRNDArray`` over
+the existing ``NDArray`` is that
+
+- memory consumption is reduced significantly
+- certain operations (e.g. matrix-vector multiplication) are much faster
+
+Meanwhile, ``CSRNDArray`` inherits competitive features from ``NDArray`` such 
as
+lazy evaluation and automatic parallelization, which are not available in the
+scientific computing python package [SciPy](https://www.scipy.org/).
+
+Apart from often queried attributes such as **ndarray.shape**, 
**ndarray.dtype** and **ndarray.context**,
+you?ll also want to query **ndarray.stype**: the storage type of the NDArray. 
For a usual dense NDArray,
+the value of stype is **"default"**. For an CSRNDArray, the value of stype is 
**"csr"**.
+
+## Prerequisites
+
+To complete this tutorial, we need:
+
+- MXNet. See the instructions for your operating system in [Setup and 
Installation](http://mxnet.io/get_started/install.html)
+- [Jupyter](http://jupyter.org/)
+```
+pip install jupyter
+```
+- Scipy - A section of this tutorial uses Scipy package in python. If you 
don't have Scipy,
+the example in that section will be ignored.
 
 Review comment:
   Suggestion: change "will" to "can". Oh, I see that you use try-exception and 
so "will" is correct. Ignore my suggestion.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nikrao commented on issue #7390: adding ranking metrics (precision/recall) at position K.

[git]

nikrao commented on issue #7390: adding ranking metrics (precision/recall) at 
position K. 
URL: https://github.com/apache/incubator-mxnet/pull/7390#issuecomment-326130469
 
 
   yes. tested that.
   
   On Wed, Aug 30, 2017 at 2:53 PM, Haibin Lin 
   wrote:
   
   > *@eric-haibin-lin* commented on this pull request.
   > --
   >
   > In python/mxnet/metric.py
   > 
   > :
   >
   > > +Parameters
   > +--
   > +labels : list of `NDArray`
   > +The labels of the data. (binary)
   > +preds : list of `NDArray`
   > +Predicted values. (float)
   > +
   > +Returns:
   > +
   > +The precision at K (float)
   > +"""
   > +check_label_shapes(labels, preds)
   > +
   > +for label, pred_label in zip(labels, preds):
   > +assert(len(pred_label.shape) <= 2), 'Predictions should be no 
more than 2 dims'
   > +pred_label = 
numpy.argsort(-pred_label.asnumpy().astype('float32'), axis=1)
   >
   > does this work when len(pred_label.shape) == 1 ??
   >
   > ?
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy commented on issue #7190: add variational autoencoder example

[git]

sandeep-krishnamurthy commented on issue #7190: add variational autoencoder 
example
URL: https://github.com/apache/incubator-mxnet/pull/7190#issuecomment-326130084
 
 
   Added licence header for now, as it was causing all other master builds to 
fail.
   Please make sure at minimum CI sanity and builds are passing for PR while we 
can fix intermittent PR build issues.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7390: adding ranking metrics (precision/recall) at position K.

[git]

eric-haibin-lin commented on a change in pull request #7390: adding ranking 
metrics (precision/recall) at position K. 
URL: https://github.com/apache/incubator-mxnet/pull/7390#discussion_r136199686
 
 

 ##
 File path: python/mxnet/metric.py
 ##
 @@ -568,6 +568,147 @@ def update(self, labels, preds):
 self.sum_metric += f1_score
 self.num_inst += 1
 
+@register
+@alias('top_k_precision')
+class TopKPrecision(EvalMetric):
+"""Computes top k precision metric.
+top k differs from regular precision in that the score is only
+computed for the top k predictions. "correct" or "wrong" entries
+outside the top k are ignored
+Parameters
+--
+top_k : int
+Whether targets are in top k predictions.
+name : str
+Name of this metric instance for display.
+output_names : list of str, or None
+Name of predictions that should be used when updating with update_dict.
+By default include all predictions.
+label_names : list of str, or None
+Name of labels that should be used when updating with update_dict.
+By default include all labels.
+
+Examples
+
+>>>ytrue = [[1.,0.,1.,0.],[0.,1.,1.,0.]]
+>>>ytrue = mx.nd.array(ytrue)
+>>>yhat = [[0.4,0.8,0.1,0.1],[0.4,0.8,0.8,0.4]]
+>>>yhat = mx.nd.array(yhat)
+>>>pre = mx.metric.create('top_k_precision',top_k=2)
+>>>pre.update(preds = [yhat], labels = [ytrue])
+>>>print pre.get()[1]
+>>> 0.75
+
+"""
+
+def __init__(self, top_k=1, name='top_k_precision',
+ output_names=None, label_names=None):
+super(TopKPrecision, self).__init__(
+name, top_k=top_k,
+output_names=output_names, label_names=label_names)
+self.top_k = top_k
+
+
+def update(self, labels, preds):
+"""Updates the internal evaluation result.
+Parameters
+--
+labels : list of `NDArray`
+The labels of the data. (binary)
+preds : list of `NDArray`
+Predicted values. (float)
+
+Returns:
+
+The precision at K (float)
+"""
+check_label_shapes(labels, preds)
+
+for label, pred_label in zip(labels, preds):
+assert(len(pred_label.shape) <= 2), 'Predictions should be no more 
than 2 dims'
+pred_label = 
numpy.argsort(-pred_label.asnumpy().astype('float32'), axis=1)
 
 Review comment:
   does this work when `len(pred_label.shape) == 1`  ??
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: Adding license header

[skm]

This is an automated email from the ASF dual-hosted git repository.

skm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 42a43ba  Adding license header
42a43ba is described below

commit 42a43ba45d1248fa75941a6b82f21840c67e52f8
Author: Sandeep Krishnamurthy 
AuthorDate: Wed Aug 30 14:51:22 2017 -0700

Adding license header
---
 example/vae/VAE.py | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/example/vae/VAE.py b/example/vae/VAE.py
index c0d5ec0..9de1abf 100644
--- a/example/vae/VAE.py
+++ b/example/vae/VAE.py
@@ -1,3 +1,21 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# pylint: skip-file
 import mxnet as mx
 import numpy as np
 import os
@@ -126,4 +144,4 @@ class VAE:
 act_z = np.tanh(decoder_z)
 decoder_x = 
np.transpose(np.dot(params['decoder_x_weight'].asnumpy(),act_z)) + 
params['decoder_x_bias'].asnumpy()
 reconstructed_x = 1/(1+np.exp(-decoder_x))
-return reconstructed_x
\ No newline at end of file
+return reconstructed_x

-- 
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" '].

[GitHub] szha opened a new pull request #7670: update zero-division/log epsilons to 1e-12

[git]

szha opened a new pull request #7670: update zero-division/log epsilons to 1e-12
URL: https://github.com/apache/incubator-mxnet/pull/7670
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy opened a new issue #7669: [Jenkins CI] Broken R unit test : test_img_seg

[git]

sandeep-krishnamurthy opened a new issue #7669: [Jenkins CI] Broken R unit test 
: test_img_seg
URL: https://github.com/apache/incubator-mxnet/issues/7669
 
 
   @thirdwing  
   
   One of the R unit tests is failing in the Jenkins CI.
   Looks like the issue is due to the installation of package failing.
   
   Master builds are failing. Can you please take a look at the issue?
   
   Build Failure Log Link -
   
https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/master/276/pipeline
   
   Error Output:
   
   Failed 
-
   1. Error: UNET (@test_img_seg.R#93) 

   trying to use CRAN without setting a mirror
   1: install.packages(new.packages) at 
R-package/tests/testthat/test_img_seg.R:93
   2: grep("^file:", contriburl)
   3: contrib.url(repos, type)
   4: stop("trying to use CRAN without setting a mirror")
   
   DONE 
===
   Error: Test failures
   Execution halted
   make: *** [rpkgtest] Error 1
   script returned exit code 2
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #7390: adding ranking metrics (precision/recall) at position K.

[git]

szha commented on issue #7390: adding ranking metrics (precision/recall) at 
position K. 
URL: https://github.com/apache/incubator-mxnet/pull/7390#issuecomment-326108993
 
 
   How about a TopKF1 class in which it provides both top k precision and 
recall? That way some counters can be shared.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhanghang1989 commented on a change in pull request #7579: style transfer example

[git]

zhanghang1989 commented on a change in pull request #7579: style transfer 
example
URL: https://github.com/apache/incubator-mxnet/pull/7579#discussion_r136180085
 
 

 ##
 File path: example/gluon/style_transfer/data.py
 ##
 @@ -0,0 +1,142 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import mxnet.gluon.data as data
+
+from PIL import Image
+import os
+import os.path
+
+IMG_EXTENSIONS = [
+'.jpg', '.JPG', '.jpeg', '.JPEG',
+'.png', '.PNG', '.ppm', '.PPM', '.bmp', '.BMP',
+]
+
+
+def is_image_file(filename):
+return any(filename.endswith(extension) for extension in IMG_EXTENSIONS)
+
+
+def find_classes(dir):
+classes = [d for d in os.listdir(dir) if os.path.isdir(os.path.join(dir, 
d))]
+classes.sort()
+class_to_idx = {classes[i]: i for i in range(len(classes))}
+return classes, class_to_idx
+
+
+def make_dataset(dir, class_to_idx):
+images = []
+dir = os.path.expanduser(dir)
+for target in sorted(os.listdir(dir)):
+d = os.path.join(dir, target)
+if not os.path.isdir(d):
+continue
+
+for root, _, fnames in sorted(os.walk(d)):
+for fname in sorted(fnames):
+if is_image_file(fname):
+path = os.path.join(root, fname)
+item = (path, class_to_idx[target])
+images.append(item)
+
+return images
+
+
+def pil_loader(path):
+# open path as file to avoid ResourceWarning 
(https://github.com/python-pillow/Pillow/issues/835)
+with open(path, 'rb') as f:
+with Image.open(f) as img:
+return img.convert('RGB')
+
+
+def accimage_loader(path):
+import accimage
+try:
+return accimage.Image(path)
+except IOError:
+# Potentially a decoding problem, fall back to PIL.Image
+return pil_loader(path)
+
+
+def default_loader(path):
+from torchvision import get_image_backend
+if get_image_backend() == 'accimage':
+return accimage_loader(path)
+else:
+return pil_loader(path)
+
+
+class ImageFolder(data.Dataset):
 
 Review comment:
   I tried that before. It is not easy to plugin the PIL image transformation
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #7579: style transfer example

[git]

piiswrong commented on a change in pull request #7579: style transfer example
URL: https://github.com/apache/incubator-mxnet/pull/7579#discussion_r136177698
 
 

 ##
 File path: example/gluon/style_transfer/data.py
 ##
 @@ -0,0 +1,142 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import mxnet.gluon.data as data
+
+from PIL import Image
+import os
+import os.path
+
+IMG_EXTENSIONS = [
+'.jpg', '.JPG', '.jpeg', '.JPEG',
+'.png', '.PNG', '.ppm', '.PPM', '.bmp', '.BMP',
+]
+
+
+def is_image_file(filename):
+return any(filename.endswith(extension) for extension in IMG_EXTENSIONS)
+
+
+def find_classes(dir):
+classes = [d for d in os.listdir(dir) if os.path.isdir(os.path.join(dir, 
d))]
+classes.sort()
+class_to_idx = {classes[i]: i for i in range(len(classes))}
+return classes, class_to_idx
+
+
+def make_dataset(dir, class_to_idx):
+images = []
+dir = os.path.expanduser(dir)
+for target in sorted(os.listdir(dir)):
+d = os.path.join(dir, target)
+if not os.path.isdir(d):
+continue
+
+for root, _, fnames in sorted(os.walk(d)):
+for fname in sorted(fnames):
+if is_image_file(fname):
+path = os.path.join(root, fname)
+item = (path, class_to_idx[target])
+images.append(item)
+
+return images
+
+
+def pil_loader(path):
+# open path as file to avoid ResourceWarning 
(https://github.com/python-pillow/Pillow/issues/835)
+with open(path, 'rb') as f:
+with Image.open(f) as img:
+return img.convert('RGB')
+
+
+def accimage_loader(path):
+import accimage
+try:
+return accimage.Image(path)
+except IOError:
+# Potentially a decoding problem, fall back to PIL.Image
+return pil_loader(path)
+
+
+def default_loader(path):
+from torchvision import get_image_backend
+if get_image_backend() == 'accimage':
+return accimage_loader(path)
+else:
+return pil_loader(path)
+
+
+class ImageFolder(data.Dataset):
 
 Review comment:
   gluon.data has this
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhreshold commented on a change in pull request #7626: modify gluon image_classification for imagenet

[git]

zhreshold commented on a change in pull request #7626: modify gluon 
image_classification for imagenet
URL: https://github.com/apache/incubator-mxnet/pull/7626#discussion_r136177564
 
 

 ##
 File path: src/io/iter_normalize.h
 ##
 @@ -144,35 +148,71 @@ class ImageNormalizeIter : public IIterator {
 float illumination =
 rand_uniform(rnd_) * param_.max_random_illumination * 2 - 
param_.max_random_illumination;
 
-if (param_.mean_r > 0.0f || param_.mean_g > 0.0f ||
-param_.mean_b > 0.0f || param_.mean_a > 0.0f) {
-  // subtract mean per channel
-  data[0] -= param_.mean_r;
-  if (data.shape_[0] >= 3) {
-data[1] -= param_.mean_g;
-data[2] -= param_.mean_b;
-  }
-  if (data.shape_[0] == 4) {
-data[3] -= param_.mean_a;
+// setup mean if not loaded from file, should happen only once
+if (!meanfile_ready_ || (meanimg_.shape_ != data.shape_)) {
+  if (param_.mean_r > 0.0f || param_.mean_g > 0.0f ||
+  param_.mean_b > 0.0f || param_.mean_a > 0.0f) {
+// setup meanimg using mean_r/g/b/a
+meanimg_.Resize(data.shape_);
+switch (static_cast(data.shape_[0])) {
+  case 4: {
+meanimg_[3] = param_.mean_a;
 
 Review comment:
   To condense operation and avoid too many branches for each input.
   the meanimg_ and stdimg_ should be modified only once for the first image. I 
would like it to happen in Init, but can't infer the shape there. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #7083: update caffe model converter for ssd models only

[git]

piiswrong commented on issue #7083: update caffe model converter for ssd models 
only
URL: https://github.com/apache/incubator-mxnet/pull/7083#issuecomment-326102075
 
 
   Could you rebase?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7089: Empty commit - DO NOT MERGE

[git]

piiswrong closed pull request #7089: Empty commit - DO NOT MERGE
URL: https://github.com/apache/incubator-mxnet/pull/7089
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7190: add variational autoencoder example

[git]

piiswrong closed pull request #7190: add variational autoencoder example
URL: https://github.com/apache/incubator-mxnet/pull/7190
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: add variational autoencoder example (#7190)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 62eaa36  add variational autoencoder example (#7190)
62eaa36 is described below

commit 62eaa36fd282d95d8e629d9d3681d336f71c514b
Author: xiaoyulu2014 
AuthorDate: Wed Aug 30 20:57:51 2017 +0100

add variational autoencoder example (#7190)

* add variational encoder example

* formatting
---
 example/vae/VAE.py|  129 +
 example/vae/VAE_example.ipynb | 1167 +
 2 files changed, 1296 insertions(+)

diff --git a/example/vae/VAE.py b/example/vae/VAE.py
new file mode 100644
index 000..c0d5ec0
--- /dev/null
+++ b/example/vae/VAE.py
@@ -0,0 +1,129 @@
+import mxnet as mx
+import numpy as np
+import os
+import logging
+
+
+class VAE:
+'''This class implements the Variational Auto Encoder'''
+
+def Bernoulli(x_hat,loss_label):
+
return(-mx.symbol.sum(mx.symbol.broadcast_mul(loss_label,mx.symbol.log(x_hat)) 
+ mx.symbol.broadcast_mul(1-loss_label,mx.symbol.log(1-x_hat)),axis=1))
+
+
+def 
__init__(self,n_latent=5,num_hidden_ecoder=400,num_hidden_decoder=400,x_train=None,x_valid=None,batch_size=100,learning_rate=0.001,weight_decay=0.01,num_epoch=100,optimizer='sgd',model_prefix=None,
 initializer = mx.init.Normal(0.01),likelihood=Bernoulli):
+
+
+self.n_latent = n_latent#dimension of the 
latent space Z
+self.num_hidden_ecoder = num_hidden_ecoder  #number of hidden 
units in the encoder
+self.num_hidden_decoder = num_hidden_decoder#number of hidden 
units in the decoder
+self.batch_size = batch_size#mini batch size
+self.learning_rate = learning_rate  #learning rate 
during training
+self.weight_decay = weight_decay#weight decay 
during training, for regulariization of parameters
+self.num_epoch = num_epoch  #total number of 
training epoch
+self.optimizer = optimizer
+
+
+
+#train the model
+self.model, self.training_loss = 
VAE.train_vae(x_train,x_valid,batch_size,n_latent,num_hidden_ecoder,num_hidden_decoder,learning_rate,weight_decay,num_epoch,optimizer,model_prefix,likelihood,initializer)
+#save model parameters (i.e. weights and biases)
+self.arg_params = self.model.get_params()[0]
+#save loss(ELBO) for the training set 
+nd_iter = 
mx.io.NDArrayIter(data={'data':x_train},label={'loss_label':x_train},batch_size 
= batch_size) 
+
+#if saved parameters, can access them at specific iteration e.g. last 
epoch using
+#   sym, arg_params, aux_params = 
mx.model.load_checkpoint(model_prefix, self.num_epoch)
+#   assert sym.tojson() == output.tojson()
+#   self.arg_params = arg_params 
+def 
train_vae(x_train,x_valid,batch_size,n_latent,num_hidden_ecoder,num_hidden_decoder,learning_rate,weight_decay,num_epoch,optimizer,model_prefix,likelihood,initializer):
+[N,features] = np.shape(x_train)  #number of examples and 
features
+
+#create data iterator to feed into NN
+nd_iter = 
mx.io.NDArrayIter(data={'data':x_train},label={'loss_label':x_train},batch_size 
= batch_size)
+if x_valid is not None:
+nd_iter_val = 
mx.io.NDArrayIter(data={'data':x_valid},label={'loss_label':x_valid},batch_size 
= batch_size)
+else:
+nd_iter_val = None
+data = mx.sym.var('data')
+loss_label = mx.sym.var('loss_label')
+
+
+#build network architucture
+encoder_h  = mx.sym.FullyConnected(data=data, 
name="encoder_h",num_hidden=num_hidden_ecoder)
+act_h = mx.sym.Activation(data=encoder_h, 
act_type="tanh",name="activation_h")
+
+
+mu  = mx.sym.FullyConnected(data=act_h, name="mu",num_hidden = 
n_latent)
+logvar  = mx.sym.FullyConnected(data=act_h, name="logvar",num_hidden = 
n_latent)
+#latent manifold
+z = mu + 
mx.symbol.broadcast_mul(mx.symbol.exp(0.5*logvar),mx.symbol.random_normal(loc=0,
 scale=1,shape=(batch_size,n_latent))) 
+decoder_z = mx.sym.FullyConnected(data=z, 
name="decoder_z",num_hidden=num_hidden_decoder)
+act_z = mx.sym.Activation(data=decoder_z, 
act_type="tanh",name="actication_z")
+
+decoder_x = mx.sym.FullyConnected(data=act_z, 
name="decoder_x",num_hidden=features)
+act_x = mx.sym.Activation(data=decoder_x, 
act_type="sigmoid",name='activation_x')
+
+KL = -0.5*mx.symbol.sum(1+logvar-pow( 
mu,2)-mx.symbol.exp(logvar),axis=1)
+
+#compute minus ELBO to minimize 
+loss = likelihood(act_x,loss_label)+KL
+output = 

[GitHub] piiswrong commented on a change in pull request #7626: modify gluon image_classification for imagenet

[git]

piiswrong commented on a change in pull request #7626: modify gluon 
image_classification for imagenet
URL: https://github.com/apache/incubator-mxnet/pull/7626#discussion_r136172290
 
 

 ##
 File path: src/io/iter_normalize.h
 ##
 @@ -144,35 +148,71 @@ class ImageNormalizeIter : public IIterator {
 float illumination =
 rand_uniform(rnd_) * param_.max_random_illumination * 2 - 
param_.max_random_illumination;
 
-if (param_.mean_r > 0.0f || param_.mean_g > 0.0f ||
-param_.mean_b > 0.0f || param_.mean_a > 0.0f) {
-  // subtract mean per channel
-  data[0] -= param_.mean_r;
-  if (data.shape_[0] >= 3) {
-data[1] -= param_.mean_g;
-data[2] -= param_.mean_b;
-  }
-  if (data.shape_[0] == 4) {
-data[3] -= param_.mean_a;
+// setup mean if not loaded from file, should happen only once
+if (!meanfile_ready_ || (meanimg_.shape_ != data.shape_)) {
+  if (param_.mean_r > 0.0f || param_.mean_g > 0.0f ||
+  param_.mean_b > 0.0f || param_.mean_a > 0.0f) {
+// setup meanimg using mean_r/g/b/a
+meanimg_.Resize(data.shape_);
+switch (static_cast(data.shape_[0])) {
+  case 4: {
+meanimg_[3] = param_.mean_a;
 
 Review comment:
   why modify meanimg and stdimg? Feels like a bad idea
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #7659: Gluon trainer updates: add learning_rate and lr_scheduler properties and add setter for learning rate

[git]

piiswrong commented on a change in pull request #7659: Gluon trainer updates: 
add learning_rate and lr_scheduler properties and add setter for learning rate
URL: https://github.com/apache/incubator-mxnet/pull/7659#discussion_r136159787
 
 

 ##
 File path: python/mxnet/gluon/trainer.py
 ##
 @@ -43,6 +43,12 @@ class Trainer(object):
 kvstore : str or KVStore
 kvstore type for multi-gpu and distributed training. See help on
 :any:`mxnet.kvstore.create` for more information.
+
+Properties
+--
+Learning_rate: float
+The learning rate of the optimizer or the learning rate of the
+LRScheduler of the optimizer if the LRScheduler is defined.
 
 Review comment:
   explain it can be accessed with trainer.learning_rate and set with 
trainer.learning_rate = xxx
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #7390: adding ranking metrics (precision/recall) at position K.

[git]

eric-haibin-lin commented on a change in pull request #7390: adding ranking 
metrics (precision/recall) at position K. 
URL: https://github.com/apache/incubator-mxnet/pull/7390#discussion_r136159509
 
 

 ##
 File path: python/mxnet/ranking_metrics.py
 ##
 @@ -0,0 +1,46 @@
+"""
 
 Review comment:
   Could you remove this file from your PR since it's all moved to metrics.py? 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed issue #6536: R Documentation hyperlinks at http://mxnet.io/api/r/index.html all lead to something other than R

[git]

piiswrong closed issue #6536: R Documentation hyperlinks at 
http://mxnet.io/api/r/index.html all lead to something other than R 
URL: https://github.com/apache/incubator-mxnet/issues/6536
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kevinthesun commented on issue #7461: [R] update tutorial link. close #6536

[git]

kevinthesun commented on issue #7461: [R] update tutorial link. close #6536
URL: https://github.com/apache/incubator-mxnet/pull/7461#issuecomment-326083576
 
 
   You need to add index information to 
https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/index.md 
just like python.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] thvasilo commented on issue #6365: mx.sym.Dropout() Segmentation fault

[git]

thvasilo commented on issue #6365: mx.sym.Dropout() Segmentation fault
URL: 
https://github.com/apache/incubator-mxnet/issues/6365#issuecomment-326082832
 
 
   @piiswrong I'm facing the same issue on a machine with CUDA 7.5, is there 
any way to use dropout with CUDA7.5 and MXNet 0.9 (or 0.10)?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #7416: update submoules with android fixes

[git]

piiswrong commented on issue #7416: update submoules with android fixes
URL: https://github.com/apache/incubator-mxnet/pull/7416#issuecomment-326081993
 
 
   Any updates?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #7261: Add grad_req parameter to Block that is passed to ParameterDict.get

[git]

piiswrong commented on issue #7261: Add grad_req parameter to Block that is 
passed to ParameterDict.get
URL: https://github.com/apache/incubator-mxnet/pull/7261#issuecomment-326081706
 
 
   closed since an alternative fix has been merged
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7261: Add grad_req parameter to Block that is passed to ParameterDict.get

[git]

piiswrong closed pull request #7261: Add grad_req parameter to Block that is 
passed to ParameterDict.get
URL: https://github.com/apache/incubator-mxnet/pull/7261
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: remove self-implemented speedometer (#7430)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new b7efd68  remove self-implemented speedometer (#7430)
b7efd68 is described below

commit b7efd68290311c45da74564d79c44e59ee8efbd6
Author: Ziyue Huang 
AuthorDate: Thu Aug 31 02:38:15 2017 +0800

remove self-implemented speedometer (#7430)
---
 example/rcnn/rcnn/core/callback.py| 35 ---
 example/rcnn/rcnn/tools/train_rcnn.py |  2 +-
 example/rcnn/rcnn/tools/train_rpn.py  |  2 +-
 example/rcnn/train_end2end.py |  2 +-
 4 files changed, 3 insertions(+), 38 deletions(-)

diff --git a/example/rcnn/rcnn/core/callback.py 
b/example/rcnn/rcnn/core/callback.py
index bacff96..06eb262 100644
--- a/example/rcnn/rcnn/core/callback.py
+++ b/example/rcnn/rcnn/core/callback.py
@@ -15,44 +15,9 @@
 # specific language governing permissions and limitations
 # under the License.
 
-import time
-import logging
 import mxnet as mx
 
 
-class Speedometer(object):
-def __init__(self, batch_size, frequent=50):
-self.batch_size = batch_size
-self.frequent = frequent
-self.init = False
-self.tic = 0
-self.last_count = 0
-
-def __call__(self, param):
-"""Callback to Show speed."""
-count = param.nbatch
-if self.last_count > count:
-self.init = False
-self.last_count = count
-
-if self.init:
-if count % self.frequent == 0:
-speed = self.frequent * self.batch_size / (time.time() - 
self.tic)
-if param.eval_metric is not None:
-name, value = param.eval_metric.get()
-s = "Epoch[%d] Batch [%d]\tSpeed: %.2f 
samples/sec\tTrain-" % (param.epoch, count, speed)
-for n, v in zip(name, value):
-s += "%s=%f,\t" % (n, v)
-logging.info(s)
-else:
-logging.info("Iter[%d] Batch [%d]\tSpeed: %.2f 
samples/sec",
- param.epoch, count, speed)
-self.tic = time.time()
-else:
-self.init = True
-self.tic = time.time()
-
-
 def do_checkpoint(prefix, means, stds):
 def _callback(iter_no, sym, arg, aux):
 arg['bbox_pred_weight_test'] = (arg['bbox_pred_weight'].T * 
mx.nd.array(stds)).T
diff --git a/example/rcnn/rcnn/tools/train_rcnn.py 
b/example/rcnn/rcnn/tools/train_rcnn.py
index c5417b3..0761891 100644
--- a/example/rcnn/rcnn/tools/train_rcnn.py
+++ b/example/rcnn/rcnn/tools/train_rcnn.py
@@ -118,7 +118,7 @@ def train_rcnn(network, dataset, image_set, root_path, 
dataset_path,
 for child_metric in [eval_metric, cls_metric, bbox_metric]:
 eval_metrics.add(child_metric)
 # callback
-batch_end_callback = callback.Speedometer(train_data.batch_size, 
frequent=frequent)
+batch_end_callback = mx.callback.Speedometer(train_data.batch_size, 
frequent=frequent, auto_reset=false)
 epoch_end_callback = callback.do_checkpoint(prefix, means, stds)
 # decide learning rate
 base_lr = lr
diff --git a/example/rcnn/rcnn/tools/train_rpn.py 
b/example/rcnn/rcnn/tools/train_rpn.py
index aaaf570..8cd0994 100644
--- a/example/rcnn/rcnn/tools/train_rpn.py
+++ b/example/rcnn/rcnn/tools/train_rpn.py
@@ -119,7 +119,7 @@ def train_rpn(network, dataset, image_set, root_path, 
dataset_path,
 for child_metric in [eval_metric, cls_metric, bbox_metric]:
 eval_metrics.add(child_metric)
 # callback
-batch_end_callback = callback.Speedometer(train_data.batch_size, 
frequent=frequent)
+batch_end_callback = mx.callback.Speedometer(train_data.batch_size, 
frequent=frequent, auto_reset=false)
 epoch_end_callback = mx.callback.do_checkpoint(prefix)
 # decide learning rate
 base_lr = lr
diff --git a/example/rcnn/train_end2end.py b/example/rcnn/train_end2end.py
index 5c94293..34fb5b3 100644
--- a/example/rcnn/train_end2end.py
+++ b/example/rcnn/train_end2end.py
@@ -126,7 +126,7 @@ def train_net(args, ctx, pretrained, epoch, prefix, 
begin_epoch, end_epoch,
 for child_metric in [rpn_eval_metric, rpn_cls_metric, rpn_bbox_metric, 
eval_metric, cls_metric, bbox_metric]:
 eval_metrics.add(child_metric)
 # callback
-batch_end_callback = callback.Speedometer(train_data.batch_size, 
frequent=args.frequent)
+batch_end_callback = mx.callback.Speedometer(train_data.batch_size, 
frequent=args.frequent, auto_reset=false)
 means = np.tile(np.array(config.TRAIN.BBOX_MEANS), config.NUM_CLASSES)
 stds = np.tile(np.array(config.TRAIN.BBOX_STDS), config.NUM_CLASSES)
 epoch_end_callback = callback.do_checkpoint(prefix, means, stds)

-- 
To stop receiving notification emails like this one, please contact

[GitHub] piiswrong closed pull request #7430: [RCNN] remove self-implemented speedometer

[git]

piiswrong closed pull request #7430: [RCNN] remove self-implemented speedometer
URL: https://github.com/apache/incubator-mxnet/pull/7430
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #7617: Fix memory leak in profiler

[git]

piiswrong commented on issue #7617: Fix memory leak in profiler
URL: https://github.com/apache/incubator-mxnet/pull/7617#issuecomment-326080879
 
 
   A shared_ptr copy has non-trivial overhead, specially since this is in 
critical path
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7609: Update ndarray.py

[git]

piiswrong closed pull request #7609: Update ndarray.py
URL: https://github.com/apache/incubator-mxnet/pull/7609
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong opened a new pull request #7668: add -1 indexing to ndarray

[git]

piiswrong opened a new pull request #7668: add -1 indexing to ndarray
URL: https://github.com/apache/incubator-mxnet/pull/7668
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on a change in pull request #7662: WIP: Isolated benchmarking support

[git]

anirudh2290 commented on a change in pull request #7662: WIP: Isolated 
benchmarking support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136152847
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -169,7 +169,7 @@ def row_sparse_pull(kv, key, data, slices, weight_array, 
priority):
 compute_only = args.compute_only
 communication_only = args.communication_only
 if (compute_only and io_only) or (compute_only and communication_only) or 
(io_only and communication_only):
-raise Exception("Only one of compute_only, io_only, communication_only 
can be set")
+assert False, "Only one of compute_only, io_only, communication_only 
can be set"
 
 Review comment:
   Thank you for the suggestion! Changed.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhanghang1989 commented on a change in pull request #7579: style transfer example

[git]

zhanghang1989 commented on a change in pull request #7579: style transfer 
example
URL: https://github.com/apache/incubator-mxnet/pull/7579#discussion_r136152593
 
 

 ##
 File path: example/gluon/style_transfer/utils.py
 ##
 @@ -0,0 +1,119 @@
+import os
+from PIL import Image
+
+import numpy as np
+import mxnet as mx
+import mxnet.ndarray as F
+
+def tensor_load_rgbimage(filename, ctx, size=None, scale=None, keep_asp=False):
 
 Review comment:
   @zhreshold is there an option to resize the image along the short edge and 
keep aspect ratio? Thanks!
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kevinthesun opened a new pull request #23: Add README file

[git]

kevinthesun opened a new pull request #23: Add README file
URL: https://github.com/apache/incubator-mxnet-site/pull/23
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] thirdwing commented on issue #7476: R-package RNN refactor

[git]

thirdwing commented on issue #7476: R-package RNN refactor
URL: https://github.com/apache/incubator-mxnet/pull/7476#issuecomment-326076547
 
 
   @jeremiedb Great work! I think this can be merged after some very minor 
tweaks:
   
   (1) you don' need any `mxnet:::`, since your code is part of this package;
   
   (2) let's remove `test_lstm.R` for now. We can add new test for RNN/LSTM 
later.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kevinthesun commented on issue #21: README.html is weird

[git]

kevinthesun commented on issue #21: README.html is weird
URL: 
https://github.com/apache/incubator-mxnet-site/issues/21#issuecomment-326076342
 
 
   That README file is for building docs. I'll upload a new README.md file.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #7667: forbid setattr type changes and enable block replacement

[git]

piiswrong commented on issue #7667: forbid setattr type changes and enable 
block replacement
URL: https://github.com/apache/incubator-mxnet/pull/7667#issuecomment-326076294
 
 
   Please add tests for these new behaviors
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7613: 1x1 convolution acceleration

[git]

piiswrong closed pull request #7613: 1x1 convolution acceleration
URL: https://github.com/apache/incubator-mxnet/pull/7613
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: 1x1 convolution acceleration (#7613)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 6562a8c  1x1 convolution acceleration (#7613)
6562a8c is described below

commit 6562a8c9d661191c2ce6ec0b2d48e16a7d818ec7
Author: chinakook 
AuthorDate: Thu Aug 31 02:18:47 2017 +0800

1x1 convolution acceleration (#7613)

* 1x1 convolution acceleration

* GEMM directly without im2col or col2im in 1x1 
convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern 
CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc.

* cpplint

* fix linalg_impl (#7611)

* fix linalg_impl

* fix

* fix

* fix

* set build status to success only after job ends (#7628)

Earlier code marks status as success initially. So any new PR shows jenkins 
status as success if we see the check mark on github. On opening the full build 
status, we see that builds haven't even started or are running.

If something fails, variable changes to failure then. So even without this 
merge, a red mark on github indicates that build has failed correctly. That 
behavior is unchanged.

* Fix build status of a test (#7629)

installs bc required by sh2ju.sh and changes the regex match to capital 
alphabet as it clashes with a warning thrown by opencv driver

* entire codebase build with mshadow_use_clas=0 (#7625)

* Update README.md (#7630)

*  unit test for csv iter, doc update for libsvmiter (#7623)

* add unit test for csv iter

* fix lint

* add libsvm to mxnet.io doc

* update libsvm doc

* gpu access of ndarray (#7496)

* gpu access of ndarray

* gpu access from C++ api

* gpu access fix

* Update c_api.cc

* Update c_api.cc

* refactor cudnn algo reg to no use string (#7561)

* refactor cudnn algo reg to no use string

* refactor ctx list

* fix

* refactor save_inputs

* Update io.md (#7634)

* fix tests (#7633)

* [build] explicitly install JDK8 (#7574)

* explicitly install openjdk8

* handle earlier version of ubuntu

* install software-properties-common

* update -y

* update commands

* Indents correction

* Add script to build doc files for all versions (#7636)

* Add script to build doc files for all versions

* Fix

* Use add versipn script of each different version

* add fashion mnist and move mnists to s3 (#7635)

* add fashion mnist and move mnists to s3

* refactor

* add doc for dataset (#7644)

* Change apache package URL to https (#7622)

* Pip installer for CoreML Converter: mxnet-to-coreml (#7624)

* Fixing CoreML converter's README: typos/grammar/etc.

* CoreML converter README update: Talk about layers first and then about 
models.

* Providing examples on converting various standard models; calling out 
issues with InceptionV3.

* Fixing CoreML converter's README: typos/grammar/etc.

* CoreML converter README update: Talk about layers first and then about 
models.

* Providing examples on converting various standard models; calling out 
issues with InceptionV3.

* Pip installer for converter: mxnet-coreml-converter.

Runs only on MacOS and python 2.7. Once inside the directory pip_package, 
user needs
to run:
python setup.py bdist_wheel
twine upload dist/*

Once uploaded it'll look like this:
https://testpypi.python.org/pypi/mxnet-coreml-converter

Also updated the README for converter to reflect this.

Note that we are going with a package per tool for the time being. Please 
leave feedback if you think it is better to adopt the policy of all the tools 
in one single package.

Unit tests continue to pass.

* More informative pypi package documentation.

* Updating MacOS in release notes to 10.11 after testing on it.

* Changing the name to mxnet-to-coreml and version to 0.1.0.

* Added license to setup.py

* Updating readme files with the correct pip package name.

* Parallelize windows unit tests of python 2 and 3 in jenkins (#7646)

* parallelize python windows tests

* reordered for clarity

* Removed asset loaded insecurely and added the asset to be loaded from the 
origin securely (#7649)

* skip failing test temporarily (#7648)

* lower really high threshold to fix test failure (#7650)

* Doc updates for install and doc generation (#7647)

* fluent (#7584)

* add 1x1 convolution to tests

* 

[GitHub] eric-haibin-lin commented on issue #7577: Sparse operators for unary and binary elemwise NDArray operators.

[git]

eric-haibin-lin commented on issue #7577: Sparse operators for unary and binary 
elemwise NDArray operators.
URL: https://github.com/apache/incubator-mxnet/pull/7577#issuecomment-326075445
 
 
   Does the code still compile?
   ```
   src/operator/nn/./../tensor/././elemwise_unary_op.h: In static member 
function 'static void mxnet::op::UnaryOp::MapToFCompute(const nnvm::NodeAttrs&, 
const mxnet::OpContext&, const std::vector&, const 
std::vector&, const std::vector&, FComputer)':
   src/operator/nn/./../tensor/././elemwise_unary_op.h:232:1: error: expected 
primary-expression before 'template'
InitStorageGeometry<1, 1>(attrs, inputs, outputs);
   ```
   Also please fix conflicting files. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #7667: forbid setattr type changes and enable block replacement

[git]

piiswrong commented on a change in pull request #7667: forbid setattr type 
changes and enable block replacement
URL: https://github.com/apache/incubator-mxnet/pull/7667#discussion_r136145851
 
 

 ##
 File path: python/mxnet/gluon/block.py
 ##
 @@ -172,9 +172,23 @@ def __repr__(self):
 
 def __setattr__(self, name, value):
 """Registers parameters."""
+
+if hasattr(self, name):
+existing = getattr(self, name)
+if not name.startswith('_') and not isinstance(value, 
type(existing)):
+raise TypeError('Changing attribute type for {name} from 
{type1} to {type2}' \
+'is not allowed.'.format(name=name,
+ type1=type(existing),
+ type2=type(value)))
+if isinstance(existing, Block):
+for i, c in enumerate(self._children):
+if c == existing:
 
 Review comment:
   is instead of ==
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: Add string interface to updater to make it consistent with kvstore (#7585)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 470b564  Add string interface to updater to make it consistent with 
kvstore (#7585)
470b564 is described below

commit 470b56437290b33fef51b067c96fdce08cefa584
Author: Haibin Lin 
AuthorDate: Wed Aug 30 11:02:00 2017 -0700

Add string interface to updater to make it consistent with kvstore (#7585)

* str kv updater draft

* backward compatibility for other languages

* add capi MXKVStoreSetUpdaterEx

* fix nightly testkvstore test

* convert c_char_p/byte to str for python3

* add key type restriction to backend

* add test to check mixed key types

* remvoe nested catch throw"
---
 include/mxnet/c_api.h |  48 +++-
 include/mxnet/kvstore.h   |  24 +-
 python/mxnet/kvstore.py   | 100 
 python/mxnet/optimizer.py |   4 +
 src/c_api/c_api.cc|  57 --
 src/kvstore/kvstore_local.h   | 140 +-
 tests/nightly/test_kvstore.py |   4 +-
 tests/python/unittest/test_kvstore.py |  96 +++
 8 files changed, 359 insertions(+), 114 deletions(-)

diff --git a/include/mxnet/c_api.h b/include/mxnet/c_api.h
index bba6190..ef9d31e 100644
--- a/include/mxnet/c_api.h
+++ b/include/mxnet/c_api.h
@@ -1602,7 +1602,7 @@ MXNET_DLL int MXKVStorePullEx(KVStoreHandle handle,
   int priority);
 
 /*!
- * \brief pull a list of (key, value) pairs from the kvstore, where each key 
is a string.
+ * \brief pull a list of (key, value) pairs from the kvstore, where each key 
is an integer.
  *The NDArray pulled back will be in row_sparse storage with only the 
specified
  *row_ids present based row_ids (others rows are zeros).
  * \param handle handle to the kvstore
@@ -1615,10 +1615,28 @@ MXNET_DLL int MXKVStorePullEx(KVStoreHandle handle,
  */
 MXNET_DLL int MXKVStorePullRowSparse(KVStoreHandle handle,
  mx_uint num,
- const char** keys,
+ const int* keys,
  NDArrayHandle* vals,
  const NDArrayHandle* row_ids,
  int priority);
+/*!
+ * \brief pull a list of (key, value) pairs from the kvstore, where each key 
is a string.
+ *The NDArray pulled back will be in row_sparse storage with only the 
specified
+ *row_ids present based row_ids (others rows are zeros).
+ * \param handle handle to the kvstore
+ * \param num the number of key-value pairs
+ * \param keys the list of keys
+ * \param vals the list of values
+ * \param row_ids the list of row_id NDArrays
+ * \param priority the priority of the action
+ * \return 0 when success, -1 when failure happens
+ */
+MXNET_DLL int MXKVStorePullRowSparseEx(KVStoreHandle handle,
+   mx_uint num,
+   const char** keys,
+   NDArrayHandle* vals,
+   const NDArrayHandle* row_ids,
+   int priority);
 
 /*!
  * \brief user-defined updater for the kvstore
@@ -1633,7 +1651,19 @@ typedef void (MXKVStoreUpdater)(int key,
 NDArrayHandle local,
 void *handle);
 /*!
- * \brief register an push updater
+ * \brief user-defined updater for the kvstore with string keys
+ * It's this updater's responsibility to delete \a recv and \a local
+ * \param the key
+ * \param recv the pushed value on this key
+ * \param local the value stored on local on this key
+ * \param handle The additional handle to the updater
+ */
+typedef void (MXKVStoreStrUpdater)(const char* key,
+   NDArrayHandle recv,
+   NDArrayHandle local,
+   void *handle);
+/*!
+ * \brief register a push updater
  * \param handle handle to the KVStore
  * \param updater udpater function
  * \param updater_handle The additional handle used to invoke the updater
@@ -1643,6 +1673,18 @@ MXNET_DLL int MXKVStoreSetUpdater(KVStoreHandle handle,
   MXKVStoreUpdater updater,
   void *updater_handle);
 /*!
+ * \brief register a push updater with int keys and one with string keys
+ * \param handle handle to the KVStore
+ * \param updater updater function with int keys
+ * \param str_updater updater function with string keys
+ * \param updater_handle The additional handle used 

[GitHub] piiswrong closed pull request #7585: Add string interface to updater to make it consistent with kvstore

[git]

piiswrong closed pull request #7585: Add string interface to updater to make it 
consistent with kvstore
URL: https://github.com/apache/incubator-mxnet/pull/7585
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #7662: WIP: Isolated benchmarking support

[git]

szha commented on a change in pull request #7662: WIP: Isolated benchmarking 
support
URL: https://github.com/apache/incubator-mxnet/pull/7662#discussion_r136143459
 
 

 ##
 File path: benchmark/python/sparse/sparse_end2end.py
 ##
 @@ -169,7 +169,7 @@ def row_sparse_pull(kv, key, data, slices, weight_array, 
priority):
 compute_only = args.compute_only
 communication_only = args.communication_only
 if (compute_only and io_only) or (compute_only and communication_only) or 
(io_only and communication_only):
-raise Exception("Only one of compute_only, io_only, communication_only 
can be set")
+assert False, "Only one of compute_only, io_only, communication_only 
can be set"
 
 Review comment:
   `assert compute_only + io_only + communication_only <= 1`
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha opened a new pull request #7667: forbid setattr type changes and enable block replacement

[git]

szha opened a new pull request #7667: forbid setattr type changes and enable 
block replacement
URL: https://github.com/apache/incubator-mxnet/pull/7667
 
 
   this change includes:
   1. prevent attributes whose names don't start with '_' from changing types
   2. properly replace _children if bot existing value and new value are blocks.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: ndarray.hpp needs to include op.h (#7495)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new ec71cff  ndarray.hpp needs to include op.h (#7495)
ec71cff is described below

commit ec71cff6bc7aa02bc6d507ad458a2806d624a9e4
Author: dtmoodie 
AuthorDate: Wed Aug 30 13:39:24 2017 -0400

ndarray.hpp needs to include op.h (#7495)

* ndarray.hpp needs to include op.h

* op.h -> operator.h
---
 cpp-package/include/mxnet-cpp/ndarray.hpp | 1 +
 1 file changed, 1 insertion(+)

diff --git a/cpp-package/include/mxnet-cpp/ndarray.hpp 
b/cpp-package/include/mxnet-cpp/ndarray.hpp
index 6bf2643..8998c0b 100644
--- a/cpp-package/include/mxnet-cpp/ndarray.hpp
+++ b/cpp-package/include/mxnet-cpp/ndarray.hpp
@@ -33,6 +33,7 @@
 #include 
 #include "dmlc/logging.h"
 #include "mxnet-cpp/ndarray.h"
+#include "mxnet-cpp/operator.h"
 
 namespace mxnet {
 namespace cpp {

-- 
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" '].

[GitHub] piiswrong closed pull request #7495: ndarray.hpp needs to include op.h

[git]

piiswrong closed pull request #7495: ndarray.hpp needs to include op.h
URL: https://github.com/apache/incubator-mxnet/pull/7495
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #7660: fix ctc on softmax grad and req option

[git]

piiswrong closed pull request #7660: fix ctc on softmax grad and req option
URL: https://github.com/apache/incubator-mxnet/pull/7660
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: fix ctc on softmax grad and req option (#7660)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 23e0a35  fix ctc on softmax grad and req option (#7660)
23e0a35 is described below

commit 23e0a3577c23c14cc77c3cb3393c5f9893373177
Author: Sheng Zha 
AuthorDate: Wed Aug 30 10:28:17 2017 -0700

fix ctc on softmax grad and req option (#7660)
---
 src/operator/contrib/ctc_loss-inl.h  | 55 
 src/operator/tensor/elemwise_unary_op.cc |  2 +-
 tests/python/gpu/test_operator_gpu.py| 20 
 3 files changed, 42 insertions(+), 35 deletions(-)

diff --git a/src/operator/contrib/ctc_loss-inl.h 
b/src/operator/contrib/ctc_loss-inl.h
index 13ce1f2..f79f0b7 100644
--- a/src/operator/contrib/ctc_loss-inl.h
+++ b/src/operator/contrib/ctc_loss-inl.h
@@ -283,16 +283,17 @@ class CTCLossOp : public Operator {
 if (!param_.use_data_lengths && !exceed_cudnn_limit) {
   cudnn_forward(ctx, s, data, costs, grad,
 _lengths, _lengths, _labels,
-max_seq_len, batch_size, alphabet_size);
+max_seq_len, batch_size, alphabet_size,
+req[ctc_loss::kGrad] != mxnet::kNullOp);
 } else {
   baidu_forward(ctx, s, data, costs, grad,
 _lengths, _lengths, _labels,
-batch_size, alphabet_size);
+batch_size, alphabet_size, req[ctc_loss::kGrad] != 
mxnet::kNullOp);
 }
 #else
 baidu_forward(ctx, s, data, costs, grad,
   _lengths, _lengths, _labels,
-  batch_size, alphabet_size);
+  batch_size, alphabet_size, req[ctc_loss::kGrad] != 
mxnet::kNullOp);
 #endif  // __CUDACC__ && CUDNN
   }
 
@@ -316,15 +317,8 @@ class CTCLossOp : public Operator {
 Tensor data_grad_computed =
 out_data[ctc_loss::kGrad].get(s);
 
-#if defined(__CUDACC__) && MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 7
-if (!param_.use_data_lengths && !exceed_cudnn_limit) {
-  cudnn_backward_extra(s, data_grad, output_grad, data_grad_computed);
-} else {
-  baidu_backward_extra(req, data_grad, output_grad, data_grad_computed);
-}
-#else
-baidu_backward_extra(req, data_grad, output_grad, data_grad_computed);
-#endif
+Assign(data_grad, req[ctc_loss::kData],
+   mshadow::expr::broadcast<1>(output_grad, data_grad.shape_) * 
data_grad_computed);
   }
 
  private:
@@ -346,7 +340,8 @@ class CTCLossOp : public Operator {
 std::vector* packed_labels,
 int max_seq_len,
 int batch_size,
-int alphabet_size) {
+int alphabet_size,
+bool req_grad) {
 using namespace mshadow;
 
 // call cudnn to calculate ctc loss
@@ -373,14 +368,14 @@ class CTCLossOp : public Operator {
   strides));
 CUDNN_CALL(cudnnGetCTCLossWorkspaceSize(s->dnn_handle_,
 prob_desc_,
-grad_desc_,
+req_grad?grad_desc_:NULL,
 packed_labels->data(),
 label_lengths->data(),
 data_lengths->data(),
 ctc_algo,
 ctc_desc_,
 _bytes));
-workspace_size = workspace_bytes/sizeof(real_t);
+workspace_size = (workspace_bytes + sizeof(real_t) - 1)/sizeof(real_t);
 
 Tensor temp_space =
   ctx.requested[ctc_loss::kTempSpace].get_space_typed(
@@ -402,19 +397,18 @@ class CTCLossOp : public Operator {
 label_lengths->data(),
 data_lengths->data(),
 costs.dptr_,
-grad_desc_,
-grad.dptr_,
+req_grad?grad_desc_:NULL,
+req_grad?grad.dptr_:NULL,
 ctc_algo,
 ctc_desc_,
 work_space.dptr_,
 workspace_bytes));
-  }
-  inline virtual void cudnn_backward_extra(mshadow::Stream* s,
-   mshadow::Tensor 
data_grad,
-   mshadow::Tensor 
output_grad,
-   mshadow::Tensor 
data_grad_computed) {

[GitHub] Godricly opened a new issue #7666: Create cublas handle failed

[git]

Godricly opened a new issue #7666: Create cublas handle failed
URL: https://github.com/apache/incubator-mxnet/issues/7666
 
 
   ## Environment info
   Operating System:
   Ubuntu? cuda8.0
   
   MXNet commit hash (`git rev-parse HEAD`):
   e05129774e76206fe890b511c346953107b05fce
   
   python2.7
   
   ## Error Message:
   Please paste the full error message, including stack trace.
   incubator-mxnet/mshadow/mshadow/./stream_gpu-inl.h:115: Check failed: err == 
CUBLAS_STATUS_SUCCESS (1 vs. 0)
   
   ## Minimum reproducible example
   if you are using your own code, please provide a short script that 
reproduces the error.
   ```python
   ctx=mx.gpu(0)
   doc_net = models.Inception3(classes=2)
   doc_net.collect_params().initialize(mx.init.Xavier(magnitude=2.24), 
ctx=ctx,verbose=True)
   ```
   
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   
   then i meet an error
   `Traceback(most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key ``conv1_1_weight`` is missing in ``args`
   
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   
   then i meet an error
   `Traceback(most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key ``conv1_1_weight`` is missing in `args`
   
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   
   then i meet an error
   `Traceback(most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key ```conv1_1_weight``` is missing in ``args``
   
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   
   then i meet an error
   `Traceback(most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key `conv1_1_weight` is missing in `args``
   
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   then i meet an error
   
   `Traceback(most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key `conv1_1_weight` is missing in `args`
   `
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   then i meet an error
   
   > `Traceback
   
(most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key `conv1_1_weight` is missing in `args`
   `
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Kaiturkey commented on issue #5124: image segmentation Error

[git]

Kaiturkey commented on issue #5124: image segmentation  Error
URL: 
https://github.com/apache/incubator-mxnet/issues/5124#issuecomment-325926819
 
 
   @zhengxiawu I meet the same bind problem,and i tried to type 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
   after 
   `load_checkpoint`
   then i meet an error
   `Traceback (most recent call last):
 File "image_segmentaion.py", line 68, in 
   main()
 File "image_segmentaion.py", line 59, in main
   exector = fcnxs.bind(ctx, fcnxs_args ,args_grad=None, grad_req="null", 
aux_states=fcnxs_args)
 File "/mxnet/python/mxnet/symbol.py", line 990, in bind
   args_handle, args = self._get_ndarray_inputs('args', args, 
listed_arguments, False)
 File "/mxnet/python/mxnet/symbol.py", line 832, in _get_ndarray_inputs
   raise ValueError('key `%s` is missing in `%s`' % (name, arg_key))
   ValueError: key `conv1_1_weight` is missing in `args`
   `
   i used FCN8s-VGG16 and gpu format params , when i run fcn_xs.py added 
   `fcnxs_args = {('arg:%s' % k) : v.as_in_context(ctx) for k, v in 
fcnxs_args.items()}`
met the similar error, is there something i did in wrong way? thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] x10000year commented on issue #7652: Need a clear and thorough tutorial for writing custom operators in c++ with nnvm

[git]

x1year commented on issue #7652: Need a clear and thorough tutorial for 
writing custom operators in c++ with nnvm
URL: 
https://github.com/apache/incubator-mxnet/issues/7652#issuecomment-325916333
 
 
   @piiswrong but the operator interface is legacy. Will it be deprecated and 
removed in near future?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] BiranLi opened a new issue #7665: Request on supporting Pooling symbol with NHWC.

[git]

BiranLi opened a new issue #7665: Request on supporting Pooling symbol with 
NHWC.
URL: https://github.com/apache/incubator-mxnet/issues/7665
 
 
   **A new feature requested as title.**
   Now, if I want to do a NHWC type pool, I should transfer it to NCHW type 
data first. It's not convenient.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] reminisce commented on issue #7652: Need a clear and thorough tutorial for writing custom operators in c++ with nnvm

[git]

reminisce commented on issue #7652: Need a clear and thorough tutorial for 
writing custom operators in c++ with nnvm
URL: 
https://github.com/apache/incubator-mxnet/issues/7652#issuecomment-325895734
 
 
   Yeah, I'm working on the tutorial pr right now. It is also expected to serve 
as a prerequisite tutorial for contributing sparse operators.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #7652: Need a clear and thorough tutorial for writing custom operators in c++ with nnvm

[git]

piiswrong commented on issue #7652: Need a clear and thorough tutorial for 
writing custom operators in c++ with nnvm
URL: 
https://github.com/apache/incubator-mxnet/issues/7652#issuecomment-325894475
 
 
   It's easier to use the legacy interface for complex operators.
   
   @reminisce Let's update the tutorial later
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services