[incubator-mxnet] branch master updated: Update mobilenetv2 symbol definition (#10595)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 301ea63  Update mobilenetv2 symbol definition (#10595)
301ea63 is described below

commit 301ea63865a6f6c3b6e7d67e07dfa837ae4ac3af
Author: Liangfu Chen 
AuthorDate: Fri Apr 27 13:55:20 2018 +0800

Update mobilenetv2 symbol definition (#10595)

* Update mobilenetv2 symbols

The pretrained model is provided in 
[this](https://github.com/liangfu/mxnet-mobilenet-v2) repository, where both 
model size and inference accuracy have been reproduced.

* update class and function names upon request

* remove camel cases in function naming

* bug fix in passing `multiplier` to `num_filter`

* update params_list

update params_list to be exactly described in the mobilenetv2 paper, which 
is published more recently in april.
---
 .../image-classification/symbols/mobilenetv2.py| 216 +
 1 file changed, 179 insertions(+), 37 deletions(-)

diff --git a/example/image-classification/symbols/mobilenetv2.py 
b/example/image-classification/symbols/mobilenetv2.py
index cfb31a8..00831ce 100644
--- a/example/image-classification/symbols/mobilenetv2.py
+++ b/example/image-classification/symbols/mobilenetv2.py
@@ -17,60 +17,202 @@
 
 # -*- coding:utf-8 -*-
 '''
-MobileNetV2, implemented in Gluon.
+MobileNetV2, implemented in built-in symbols.
 
 Reference:
 Inverted Residuals and Linear Bottlenecks:
 Mobile Networks for Classification, Detection and Segmentation
 https://arxiv.org/abs/1801.04381
 '''
-__author__ = 'dwSun'
-__date__ = '18/1/31'
+__author__ = 'liangfu'
+__date__ = '18/4/3'
 
 import mxnet as mx
 
-from mxnet.gluon.model_zoo.vision.mobilenet import MobileNetV2
+def relu6(data, prefix):
+return mx.sym.clip(data,0,6,name='%s-relu6'%prefix)
 
+def shortcut(data_in, data_residual, prefix):
+out=mx.sym.elemwise_add(data_in, data_residual, name='%s-shortcut'%prefix)
+return out
 
-__all__ = ['MobileNetV2', 'get_symbol']
+def mobilenet_unit(data, num_filter=1, kernel=(1, 1), stride=(1, 1), pad=(0, 
0), num_group=1, if_act=True, prefix=''):
+conv = mx.sym.Convolution(
+data=data,
+num_filter=num_filter,
+kernel=kernel,
+num_group=num_group,
+stride=stride,
+pad=pad,
+no_bias=True,
+name='%s-conv2d'%prefix)
+bn = mx.sym.BatchNorm(data=conv, name='%s-batchnorm'%prefix, 
fix_gamma=False, use_global_stats=False, eps=1e-5)
+if if_act:
+act = relu6(bn, prefix)
+return act
+else:
+return bn
 
+def inverted_residual_unit(data, num_in_filter, num_filter, ifshortcut, 
stride, kernel, pad, expansion_factor, prefix):
+num_expfilter = int(round(num_in_filter*expansion_factor))
 
-def get_symbol(num_classes=1000, multiplier=1.0, ctx=mx.cpu(), **kwargs):
-r"""MobileNetV2 model from the
-`"Inverted Residuals and Linear Bottlenecks:
-  Mobile Networks for  Classification, Detection and Segmentation"
-`_ paper.
+channel_expand = mobilenet_unit(
+data=data,
+num_filter=num_expfilter,
+kernel=(1,1),
+stride=(1,1),
+pad=(0,0),
+num_group=1,
+if_act=True,
+prefix='%s-exp'%prefix,
+)
+bottleneck_conv = mobilenet_unit(
+data= channel_expand,
+num_filter=num_expfilter,
+stride=stride,
+kernel=kernel,
+pad=pad,
+num_group=num_expfilter,
+if_act=True,
+prefix='%s-depthwise'%prefix,
+)
+linear_out = mobilenet_unit(
+data=bottleneck_conv,
+num_filter=num_filter,
+kernel=(1, 1),
+stride=(1, 1),
+pad=(0, 0),
+num_group=1,
+if_act=False,
+prefix='%s-linear'%prefix
+)
+if ifshortcut:
+out = shortcut(
+data_in=data,
+data_residual=linear_out,
+prefix=prefix,
+) 
+return out
+else:
+return linear_out
 
-Parameters
---
-num_classes : int, default 1000
-Number of classes for the output layer.
-multiplier : float, default 1.0
-The width multiplier for controling the model size. The actual number 
of channels
-is equal to the original channel size multiplied by this multiplier.
-ctx : Context, default CPU
-The context in which to initialize the model weights.
-"""
-net = MobileNetV2(multiplier=multiplier, classes=num_classes, **kwargs)
-net.initialize(ctx=ctx, init=mx.init.Xavier())
-net.hybridize()
+def inverted_residual_blocks(data, in_c, t, c, n, s, prefix):
+first_block = inverted_residual_unit(
+data=data,
+num_in_filter=in_c,
+num_filter=c,
+ 

[GitHub] szha closed pull request #10595: Update mobilenetv2 symbol definition

[GitBox]

szha closed pull request #10595: Update mobilenetv2 symbol definition
URL: https://github.com/apache/incubator-mxnet/pull/10595
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/example/image-classification/symbols/mobilenetv2.py 
b/example/image-classification/symbols/mobilenetv2.py
index cfb31a8fe2d..00831ce4023 100644
--- a/example/image-classification/symbols/mobilenetv2.py
+++ b/example/image-classification/symbols/mobilenetv2.py
@@ -17,60 +17,202 @@
 
 # -*- coding:utf-8 -*-
 '''
-MobileNetV2, implemented in Gluon.
+MobileNetV2, implemented in built-in symbols.
 
 Reference:
 Inverted Residuals and Linear Bottlenecks:
 Mobile Networks for Classification, Detection and Segmentation
 https://arxiv.org/abs/1801.04381
 '''
-__author__ = 'dwSun'
-__date__ = '18/1/31'
+__author__ = 'liangfu'
+__date__ = '18/4/3'
 
 import mxnet as mx
 
-from mxnet.gluon.model_zoo.vision.mobilenet import MobileNetV2
+def relu6(data, prefix):
+return mx.sym.clip(data,0,6,name='%s-relu6'%prefix)
 
+def shortcut(data_in, data_residual, prefix):
+out=mx.sym.elemwise_add(data_in, data_residual, name='%s-shortcut'%prefix)
+return out
 
-__all__ = ['MobileNetV2', 'get_symbol']
+def mobilenet_unit(data, num_filter=1, kernel=(1, 1), stride=(1, 1), pad=(0, 
0), num_group=1, if_act=True, prefix=''):
+conv = mx.sym.Convolution(
+data=data,
+num_filter=num_filter,
+kernel=kernel,
+num_group=num_group,
+stride=stride,
+pad=pad,
+no_bias=True,
+name='%s-conv2d'%prefix)
+bn = mx.sym.BatchNorm(data=conv, name='%s-batchnorm'%prefix, 
fix_gamma=False, use_global_stats=False, eps=1e-5)
+if if_act:
+act = relu6(bn, prefix)
+return act
+else:
+return bn
 
+def inverted_residual_unit(data, num_in_filter, num_filter, ifshortcut, 
stride, kernel, pad, expansion_factor, prefix):
+num_expfilter = int(round(num_in_filter*expansion_factor))
 
-def get_symbol(num_classes=1000, multiplier=1.0, ctx=mx.cpu(), **kwargs):
-r"""MobileNetV2 model from the
-`"Inverted Residuals and Linear Bottlenecks:
-  Mobile Networks for  Classification, Detection and Segmentation"
-`_ paper.
+channel_expand = mobilenet_unit(
+data=data,
+num_filter=num_expfilter,
+kernel=(1,1),
+stride=(1,1),
+pad=(0,0),
+num_group=1,
+if_act=True,
+prefix='%s-exp'%prefix,
+)
+bottleneck_conv = mobilenet_unit(
+data= channel_expand,
+num_filter=num_expfilter,
+stride=stride,
+kernel=kernel,
+pad=pad,
+num_group=num_expfilter,
+if_act=True,
+prefix='%s-depthwise'%prefix,
+)
+linear_out = mobilenet_unit(
+data=bottleneck_conv,
+num_filter=num_filter,
+kernel=(1, 1),
+stride=(1, 1),
+pad=(0, 0),
+num_group=1,
+if_act=False,
+prefix='%s-linear'%prefix
+)
+if ifshortcut:
+out = shortcut(
+data_in=data,
+data_residual=linear_out,
+prefix=prefix,
+) 
+return out
+else:
+return linear_out
 
-Parameters
---
-num_classes : int, default 1000
-Number of classes for the output layer.
-multiplier : float, default 1.0
-The width multiplier for controling the model size. The actual number 
of channels
-is equal to the original channel size multiplied by this multiplier.
-ctx : Context, default CPU
-The context in which to initialize the model weights.
-"""
-net = MobileNetV2(multiplier=multiplier, classes=num_classes, **kwargs)
-net.initialize(ctx=ctx, init=mx.init.Xavier())
-net.hybridize()
+def inverted_residual_blocks(data, in_c, t, c, n, s, prefix):
+first_block = inverted_residual_unit(
+data=data,
+num_in_filter=in_c,
+num_filter=c,
+ifshortcut=False,
+stride=(s,s),
+kernel=(3,3),
+pad=(1,1),
+expansion_factor=t,
+prefix='%s-block0'%prefix
+)
 
-data = mx.sym.var('data')
-out = net(data)
-sym = mx.sym.SoftmaxOutput(out, name='softmax')
-return sym
+last_residual_block = first_block
+last_c = c
 
+for i in range(1,n):
+last_residual_block = inverted_residual_unit(
+data=last_residual_block,
+num_in_filter=last_c,
+num_filter=c,
+ifshortcut=True,
+stride=(1,1),
+kernel=(3,3),
+pad=(1,1),
+expansion_factor=t,
+prefix='%s-block%d'%(prefix, i)
+)
+return last_residual_block
 
-def plot_net():
-"""
-

[GitHub] chinakook commented on issue #10558: NNVM build failed in the newst mxnet version

[GitBox]

chinakook commented on issue #10558: NNVM build failed in the newst mxnet 
version
URL: 
https://github.com/apache/incubator-mxnet/issues/10558#issuecomment-384871407
 
 
   @rahul003 However, the version of gcc6 in Ubuntu 16.04 is 6.0.1


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on issue #10694: Importing an ONNX model (from model zoo) to MXNet errors out

[GitBox]

lupesko commented on issue #10694: Importing an ONNX model (from model zoo) to 
MXNet errors out
URL: 
https://github.com/apache/incubator-mxnet/issues/10694#issuecomment-384870900
 
 
   Thanks @spidyDev.
   Closing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko closed issue #10694: Importing an ONNX model (from model zoo) to MXNet errors out

[GitBox]

lupesko closed issue #10694: Importing an ONNX model (from model zoo) to MXNet 
errors out
URL: https://github.com/apache/incubator-mxnet/issues/10694
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jakirkham commented on issue #9205: compile error with clang

[GitBox]

jakirkham commented on issue #9205: compile error with clang
URL: 
https://github.com/apache/incubator-mxnet/issues/9205#issuecomment-384870346
 
 
   Clang 3.7+ has OpenMP support. Though it is a separate library and not all 
installs ship it (e.g. Apple's copy for instance). However it is trivial to 
build yourself. Also you can get copies of the complete Clang toolchain from 
your package manager of choice or Anaconda if you use that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jakirkham commented on issue #3441: Install mxnet into anaconda's python

[GitBox]

jakirkham commented on issue #3441: Install mxnet into anaconda's python
URL: 
https://github.com/apache/incubator-mxnet/issues/3441#issuecomment-384869278
 
 
   We at [conda-forge]( https://conda-forge.org/ ) would be happy to help you 
with the review of an `mxnet` recipe for `conda` if someone would be interested 
in submitting one. Issue ( 
https://github.com/conda-forge/staged-recipes/issues/4447 ) would be a great 
place to discuss this if someone would like to step forward.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] hbsun2113 commented on issue #10699: Is the "gradient compression" only suitable for range_pull?

[GitBox]

hbsun2113 commented on issue #10699: Is the "gradient compression" only 
suitable for range_pull?
URL: 
https://github.com/apache/incubator-mxnet/issues/10699#issuecomment-384867931
 
 
   @rahul003
   
![image](https://user-images.githubusercontent.com/27994442/39345992-6d182c42-4a1e-11e8-91bc-2cc3bfb74f40.png)
   I think it is the range_push.Can mxnet push or pull by hash kv?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] reminisce commented on issue #10710: Add cuda version check to skip building quantization ops for versions less than 8.0

[GitBox]

reminisce commented on issue #10710: Add cuda version check to skip building 
quantization ops for versions less than 8.0
URL: https://github.com/apache/incubator-mxnet/pull/10710#issuecomment-384866937
 
 
   @marcoabreu I launched a Ubuntu 14.04 instance and installed cuda-7.5 and 
cudnn-6. The build can pass successfully with this PR. Unit tests other than 
quantization_gpu can pass as well which is as expected.
   
   Please help to review: @szha @piiswrong @anirudh2290 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] chinakook commented on issue #10558: NNVM build failed in the newst mxnet version

[GitBox]

chinakook commented on issue #10558: NNVM build failed in the newst mxnet 
version
URL: 
https://github.com/apache/incubator-mxnet/issues/10558#issuecomment-384863876
 
 
   I'll try later


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] chinakook commented on issue #10558: NNVM build failed in the newst mxnet version

[GitBox]

chinakook commented on issue #10558: NNVM build failed in the newst mxnet 
version
URL: 
https://github.com/apache/incubator-mxnet/issues/10558#issuecomment-384863500
 
 
   i7-6700k. I've tested on Ubuntu 17.10 and 18.04, both failed without 
USE_F16C=0.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #10716: question: how to speed up the process of feeding data.

[GitBox]

rahul003 commented on issue #10716: question: how to speed up the process of 
feeding data.
URL: 
https://github.com/apache/incubator-mxnet/issues/10716#issuecomment-384862573
 
 
   Are you using gluon or the symbolic interface?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #10716: question: how to speed up the process of feeding data.

[GitBox]

rahul003 commented on issue #10716: question: how to speed up the process of 
feeding data.
URL: 
https://github.com/apache/incubator-mxnet/issues/10716#issuecomment-384862573
 
 
   Are you using the gluon or symbolic interface?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #10558: NNVM build failed in the newst mxnet version

[GitBox]

rahul003 commented on issue #10558: NNVM build failed in the newst mxnet version
URL: 
https://github.com/apache/incubator-mxnet/issues/10558#issuecomment-384862516
 
 
   What CPU do you have? Could you run the diagnose script mentioned in the 
issue and paste the output to the issue?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10591: handle inplace in mkldnn FallBackCompute

[GitBox]

ashokei commented on issue #10591: handle inplace in mkldnn FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#issuecomment-384854306
 
 
   @zheng-da added unittest for this, please accept review if ok , thanks


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] liangfu commented on issue #10595: Update mobilenetv2 symbol definition

[GitBox]

liangfu commented on issue #10595: Update mobilenetv2 symbol definition
URL: https://github.com/apache/incubator-mxnet/pull/10595#issuecomment-384852497
 
 
   I think this PR is ready to merge.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] GoodJoey opened a new issue #10716: question: how to speed up the process of feeding data.

[GitBox]

GoodJoey opened a new issue #10716: question: how to speed up the process of 
feeding data.
URL: https://github.com/apache/incubator-mxnet/issues/10716
 
 
   hi, i tried to do the training on the 8 gpus server, but it turns out that 
it's not faster than when using 4 gpus.
   The most likely reason is the GPUs are faster than the processes that load 
and prepare the data, because i observed the gpu's utility goes down to 0% time 
by time.
   is there any ways to speed up the data feeding process? like use multiple 
cpus to do the data loading?
   Thanks.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wkcn commented on issue #10242: [MXNET-137]fix parameters name inconsistent for Proposal OP and Multi Proposal OP

[GitBox]

wkcn commented on issue #10242: [MXNET-137]fix parameters name inconsistent for 
Proposal OP and Multi Proposal OP
URL: https://github.com/apache/incubator-mxnet/pull/10242#issuecomment-384836993
 
 
   @piiswrong 
   Hi! 
   The document shows that [the sym 
version](https://mxnet.incubator.apache.org/api/python/symbol/contrib.html?highlight=proposal#mxnet.symbol.contrib.Proposal)
 of Proposal OP uses **cls_score**.
   However, [the Faster R-CNN 
example](https://github.com/apache/incubator-mxnet/blob/master/example/rcnn/rcnn/symbol/symbol_resnet.py#L193)
 uses **cls_prob**.
   
   If making them consistent by using the sym version(cls_score), the old 
example and other detection 
projects([Deformable-ConvNets](https://github.com/msracver/Deformable-ConvNets/blob/master/faster_rcnn/symbols/resnet_v1_101_rcnn.py#L741),
 
[mx-rcnn](https://github.com/precedenceguo/mx-rcnn/blob/master/rcnn/symbol/symbol_resnet.py#L100))
 will be an error of parameter.
   
   I think using `cls_prob` is better. 
[mxnet.ndarray.contrib.MultiBoxDetection](https://mxnet.incubator.apache.org/api/python/ndarray/contrib.html?highlight=multibox#mxnet.ndarray.contrib.MultiBoxDetection)
 uses `cls_prob` too.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] olympian94 commented on issue #10505: Profiler profiler shoudl collect call durations not just timestamps

[GitBox]

olympian94 commented on issue #10505: Profiler  profiler shoudl collect call 
durations not just timestamps
URL: 
https://github.com/apache/incubator-mxnet/issues/10505#issuecomment-384836414
 
 
   Multiple files? I don't get it. The profiler puts all event traces in one 
json file. And yes this mechanism is inefficient in that the output file grows 
into GBs in size for realistic models however the authors warn about that in 
the documentation. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yajiedesign commented on a change in pull request #10629: [MXNET-343]fix Mkldnn with msvc

[GitBox]

yajiedesign commented on a change in pull request #10629: [MXNET-343]fix Mkldnn 
with msvc
URL: https://github.com/apache/incubator-mxnet/pull/10629#discussion_r184568865
 
 

 ##
 File path: cmake/FirstClassLangCuda.cmake
 ##
 @@ -126,7 +126,7 @@ endif ()
 function(mshadow_select_nvcc_arch_flags out_variable)
 
   set(CUDA_ARCH_LIST "Auto" CACHE STRING "Select target NVIDIA GPU 
achitecture.")
-  set_property( CACHE CUDA_ARCH_LIST PROPERTY STRINGS "" "All" "Common" 
${CUDA_KNOWN_GPU_ARCHITECTURES} )
+  set_property( CACHE CUDA_ARCH_LIST PROPERTY STRINGS "" "Auto" "All" "Common" 
${CUDA_KNOWN_GPU_ARCHITECTURES} )
 
 Review comment:
   no.default behaviour always is "Auto",this change allow users to choose 
"Auto"


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yajiedesign commented on a change in pull request #10697: [MXNET-360]auto convert str to bytes in img.imdecode when py3

[GitBox]

yajiedesign commented on a change in pull request #10697: [MXNET-360]auto 
convert str to bytes in img.imdecode when py3
URL: https://github.com/apache/incubator-mxnet/pull/10697#discussion_r184568671
 
 

 ##
 File path: python/mxnet/image/image.py
 ##
 @@ -132,6 +134,9 @@ def imdecode(buf, *args, **kwargs):
 
 """
 if not isinstance(buf, nd.NDArray):
+if sys.version_info[0] == 3:
+if isinstance(buf, str):
+buf = bytes(buf, "ascii")
 
 Review comment:
mx.base.py_str is 'utf8' encoding,I'm not sure it can work.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yajiedesign commented on issue #10697: [MXNET-360]auto convert str to bytes in img.imdecode when py3

[GitBox]

yajiedesign commented on issue #10697: [MXNET-360]auto convert str to bytes in 
img.imdecode when py3
URL: https://github.com/apache/incubator-mxnet/pull/10697#issuecomment-384830401
 
 
   @piiswrong np.frombuffer receive buffer like param,but in python3 str not is 
buffer like.will trigger TypeError.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yajiedesign commented on issue #10697: [MXNET-360]auto convert str to bytes in img.imdecode when py3

[GitBox]

yajiedesign commented on issue #10697: [MXNET-360]auto convert str to bytes in 
img.imdecode when py3
URL: https://github.com/apache/incubator-mxnet/pull/10697#issuecomment-384830401
 
 
   @piiswrong np.frombuffer receive buffer like param,but in python3 str nor is 
buffer like.will trigger TypeError.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] solin319 closed pull request #8423: Re-implement segnet in MXnet

[GitBox]

solin319 closed pull request #8423: Re-implement segnet in MXnet
URL: https://github.com/apache/incubator-mxnet/pull/8423
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/example/segnet/README.md b/example/segnet/README.md
new file mode 100644
index 000..efd4d9bbf17
--- /dev/null
+++ b/example/segnet/README.md
@@ -0,0 +1,108 @@
+# SegNet
+
+SegNet is a deep encoder-decoder architecture for multi-class pixelwise 
segmentation researched and developed by members of the [Computer Vision and 
Robotics Group](http://mi.eng.cam.ac.uk/Main/CVR) at the University of 
Cambridge, UK.
+
+This mxnet version reference a caffe version 
https://github.com/alexgkendall/SegNet-Tutorial .
+
+The segnet_basic and segnet networks were included in this mxnet version.
+
+## Requirement
+
+Need python package Pillow.
+```
+pip install Pillow
+```
+Build MXNet with new pooling and upsampling operators.
+```
+# copy new operators to src/operator/
+cp segnet/op/* incubator-mxnet/src/operator/
+# rebuild MXNet from source
+cd incubator-mxnet/
+make
+cd python/
+python setup.py install
+```
+
+## Dataset
+
+The model can be trained for road scene understanding using the [CamVid 
dataset](http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/). The 
Cambridge-driving Labeled Video Database (CamVid) is the first collection of 
videos with object class semantic labels, complete with metadata. The database 
provides ground truth labels that associate each pixel with one of [32 semantic 
classes](http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/#ClassLabels).
+
+This dataset is small, consisting of 367 training and 233 testing RGB images 
(day and dusk scenes) at
+360*480 resolution. The challenge is to segment 11 classes such
+as road, building, cars, pedestrians, signs, poles, side-walk etc.
+
+## Train
+
+- Getting Started
+
+  Install python package `Pillow` (required by `image_segment.py`).
+
+```
+[sudo] pip install Pillow
+```
+
+- train model
+```
+python train_segnet.py \
+--gpus=0,1,2,3 \
+--lr=0.005 \
+--wd=0.0005 \
+--network=segnet_basic \
+--batch-size=12
+```
+
+​  The output log may look like this
+```
+Epoch[132] Batch [5]Speed: 14.15 samples/secaccuracy=0.903786
+Epoch[132] Batch [10]   Speed: 14.80 samples/secaccuracy=0.913140
+Epoch[132] Batch [15]   Speed: 13.62 samples/secaccuracy=0.909260
+Epoch[132] Batch [20]   Speed: 14.09 samples/secaccuracy=0.893901
+Epoch[132] Batch [25]   Speed: 14.74 samples/secaccuracy=0.911765
+Epoch[132] Train-accuracy=0.906672
+Epoch[132] Time cost=25.005
+Epoch[132] Validation-accuracy=0.819186
+
+```
+- Using the pre-trained model for image segmentation
+
+  It load parameters from model_prefix and load_epoch as normal.
+
+  We can also load convolution parameters from vgg16 model by use 
model_prefix_for_vgg16 and load_epoch_for_vgg16.
+- The accuracy can reach 0.84 with segnet_basic. When train segnet with vgg 
pre-trained parameters, the accuracy will reach about 0.89 .
+- Use bayesian+segnet+vgg_pretrained to train the model in the first 290 epoch 
and use segnet to train the last 
+10 epoch, the accuracy can reach 0.91 .
+## Score
+
+- score the model
+
+```
+python score.py --batch-size=1 --load_epoch=270 --score_file=test.txt
+```
+
+The output log may look like this. Each line shows the global segment accuracy 
in one picture. The segment result can be saved in a package named "res_pic".
+
+```
+INFO:root:('accuracy', 0.80057751076185835)
+INFO:root:('accuracy', 0.80030805232087987)
+INFO:root:('accuracy', 0.80030805232087987)
+INFO:root:('accuracy', 0.80014792169887283)
+INFO:root:('accuracy', 0.80014792169887283)
+INFO:root:('accuracy', 0.80037548100048095)
+INFO:root:('accuracy', 0.80037548100048095)
+
+```
+
+## Segment one pic
+
+```
+python segment_one_pic.py \
+--image_name=0016E5_04530.png \
+--label_name=0016E5_04530.png \
+--load_epoch=270 \
+```
+
+The program will output segment result and label in a package named "res_pic".
+
+The left one is a label picture with 11 classes. The right one below is a 
segment result.
+
+![label2](https://user-images.githubusercontent.com/13029886/32312590-9120270e-bfd9-11e7-9fdb-de29aece2422.png)
 
![res2](https://user-images.githubusercontent.com/13029886/32312591-9159ea7a-bfd9-11e7-8658-e3fa1ce90bf2.png)
diff --git a/example/segnet/__init__.py b/example/segnet/__init__.py
new file mode 100644
index 000..e69de29bb2d
diff --git a/example/segnet/common/__init__.py 
b/example/segnet/common/__init__.py
new file mode 100644
index 000..e69de29bb2d
diff --git a/example/segnet/common/contrib_metrics.py 
b/example/segnet/common/contrib_metrics.py
new file mode 100644
index 000..5bbf182ec28
--- /dev/null
+++ 

[GitHub] solin319 closed pull request #10234: [MXNET-135] mx.image.imread support s3

[GitBox]

solin319 closed pull request #10234: [MXNET-135] mx.image.imread support s3
URL: https://github.com/apache/incubator-mxnet/pull/10234
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/io/image_io.cc b/src/io/image_io.cc
index f6183a12c2d..04e8a34e477 100644
--- a/src/io/image_io.cc
+++ b/src/io/image_io.cc
@@ -23,6 +23,7 @@
  * \brief Optimizer operators
  * \author Junyuan Xie
  */
+#include <../src/io/filesys.h>
 #include 
 #include 
 #include 
@@ -217,17 +218,17 @@ void Imread(const nnvm::NodeAttrs& attrs,
 std::vector* outputs) {
 #if MXNET_USE_OPENCV
   const auto& param = nnvm::get(attrs.parsed);
-
-  std::ifstream file(param.filename, std::ios::binary | std::ios::ate);
-  // if file is not open we get bad alloc after tellg
-  CHECK(file.is_open()) << "Imread: '" << param.filename
-  << "' couldn't open file: " << strerror(errno);
-  size_t fsize = file.tellg();
-  file.seekg(0, std::ios::beg);
+  dmlc::io::URI path(param.filename.c_str());
+  dmlc::io::FileSystem *fs = dmlc::io::FileSystem::GetInstance(path);
+  CHECK(fs != NULL) << "Imread: '" << param.filename
+<< "' couldn't open file: " << strerror(errno);
+  dmlc::io::FileInfo info = fs->GetPathInfo(path);
+  size_t fsize = info.size;
+  std::unique_ptr fi(fs->Open(path, "r", true));
   std::shared_ptr buff(new uint8_t[fsize], 
std::default_delete());
-  file.read(reinterpret_cast(buff.get()), fsize);
-  CHECK(file.good()) << "Failed reading image file: '" << param.filename << "' 
"
-<< strerror(errno);
+  size_t size = fi.get()->Read(reinterpret_cast(buff.get()), fsize);
+  CHECK(fsize == size) << "Failed reading image file: '" << param.filename << 
"' "
+<< strerror(errno);
 
   TShape oshape(3);
   oshape[2] = param.flag == 0 ? 1 : 3;


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] solin319 closed pull request #10381: support profile can be saved to s3

[GitBox]

solin319 closed pull request #10381: support profile can be saved to s3 
URL: https://github.com/apache/incubator-mxnet/pull/10381
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/profiler/profiler.cc b/src/profiler/profiler.cc
index f2d14cf2729..611eef4a0e1 100644
--- a/src/profiler/profiler.cc
+++ b/src/profiler/profiler.cc
@@ -95,6 +95,10 @@ Profiler::~Profiler() {
 thread_group_->join_all();
 thread_group_.reset();
   }
+  delete file;
+  delete fo;
+  file = nullptr;
+  fo = nullptr;
 }
 
 Profiler* Profiler::Get(std::shared_ptr *sp) {
@@ -137,6 +141,8 @@ void Profiler::SetConfig(int mode,
   if (!this->filename_.empty()) {
 ::unlink(this->filename_.c_str());
   }
+  this->fo = dmlc::Stream::Create(this->filename_.c_str(), "w");
+  this->file = new dmlc::ostream(this->fo);
   SetContinuousProfileDump(continuous_dump, dump_period);
   // Adjust whether storing aggregate stats as necessary
   if (aggregate_stats) {
@@ -171,17 +177,11 @@ void Profiler::DumpProfile(bool peform_cleanup) {
   if (peform_cleanup) {
 SetContinuousProfileDump(false, 1.0f);
   }
-  std::ofstream file;
   const bool first_pass = ++profile_dump_count_ == 1;
   const bool last_pass = peform_cleanup || !continuous_dump_;
-  if (!first_pass && continuous_dump_) {
-file.open(filename_, std::ios::app|std::ios::out);
-  } else {
-file.open(filename_, std::ios::trunc|std::ios::out);
-  }
   if (first_pass || !continuous_dump_) {
-file << "{" << std::endl;
-file << "\"traceEvents\": [" << std::endl;
+*file << "{" << std::endl;
+*file << "\"traceEvents\": [" << std::endl;
   }
 
   const size_t dev_num = DeviceCount();
@@ -189,10 +189,10 @@ void Profiler::DumpProfile(bool peform_cleanup) {
   if (first_pass) {
 for (uint32_t pid = 0; pid < dev_num; ++pid) {
if (pid) {
- file << ",\n";
+ *file << ",\n";
}
   const DeviceStats  = profile_stat[pid];
-  this->EmitPid(, d.dev_name_, pid);
+  this->EmitPid(file, d.dev_name_, pid);
   process_ids_.emplace(pid);
 }
   }
@@ -208,8 +208,8 @@ void Profiler::DumpProfile(bool peform_cleanup) {
   CHECK_NOTNULL(_opr_stat);
   std::unique_ptr opr_stat(_opr_stat);  // manage lifecycle
   opr_stat->process_id_ = i;  // lie and set process id to be the device 
number
-  file << ",\n" << std::endl;
-  opr_stat->EmitEvents();
+  *file << ",\n" << std::endl;
+  opr_stat->EmitEvents(file);
   ++num_records_emitted_;
   if (ptr_aggregate_stats) {
 ptr_aggregate_stats->OnProfileStat(*_opr_stat);
@@ -221,7 +221,7 @@ void Profiler::DumpProfile(bool peform_cleanup) {
   ProfileStat *_profile_stat;
   while (general_stats_.opr_exec_stats_->try_dequeue(_profile_stat)) {
 CHECK_NOTNULL(_profile_stat);
-file << ",";
+*file << ",";
 std::unique_ptr profile_stat(_profile_stat);  // manage 
lifecycle
 CHECK_NE(profile_stat->categories_.c_str()[0], '\0') << "Category must be 
set";
 // Currently, category_to_pid_ is only accessed here, so it is protected 
by this->m_ above
@@ -231,12 +231,12 @@ void Profiler::DumpProfile(bool peform_cleanup) {
   const size_t this_pid = hash_fn(profile_stat->categories_.c_str());
   iter = 
category_to_pid_.emplace(std::make_pair(profile_stat->categories_.c_str(),
  this_pid)).first;
-  EmitPid(, profile_stat->categories_.c_str(), iter->second);
-  file << ",\n";
+  EmitPid(file, profile_stat->categories_.c_str(), iter->second);
+  *file << ",\n";
 }
 profile_stat->process_id_ = iter->second;
-file << std::endl;
-profile_stat->EmitEvents();
+*file << std::endl;
+profile_stat->EmitEvents(file);
 ++num_records_emitted_;
 if (ptr_aggregate_stats) {
   ptr_aggregate_stats->OnProfileStat(*profile_stat);
@@ -244,10 +244,11 @@ void Profiler::DumpProfile(bool peform_cleanup) {
   }
 
   if (last_pass) {
-file << "\n" << std::endl;
-file << "]," << std::endl;
-file << "\"displayTimeUnit\": \"ms\"" << std::endl;
-file << "}" << std::endl;
+*file << "\n" << std::endl;
+*file << "]," << std::endl;
+*file << "\"displayTimeUnit\": \"ms\"" << std::endl;
+*file << "}" << std::endl;
+(*file).set_stream(nullptr);
   }
   enable_output_ = continuous_dump_ && !last_pass;  // If we're appending, 
then continue.
 // Otherwise, profiling 
stops.
diff --git a/src/profiler/profiler.h b/src/profiler/profiler.h
index b8d0e8ef340..867185be25e 100644
--- a/src/profiler/profiler.h
+++ b/src/profiler/profiler.h
@@ -481,6 +481,10 @@ class Profiler {
   std::shared_ptr thread_group_ = 

[GitHub] luoyetx commented on issue #10695: SymbolBlock has no `_reg_params` which cause save/load params fails

[GitBox]

luoyetx commented on issue #10695: SymbolBlock has no `_reg_params` which cause 
save/load params fails
URL: 
https://github.com/apache/incubator-mxnet/issues/10695#issuecomment-384826547
 
 
   @chinakook The model doesn't even save the parameters of symbol blocks, how 
to load by this function. I know `net.collect_params().save` can save the 
parameters of symbol blocks. But Why `net.save_params` and 
`net.collect_params().save` gives different behave, It's a **Bug** after this 
[PR](https://github.com/apache/incubator-mxnet/pull/10511) merged.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] bputrycz opened a new issue #10715: Wrong params file is not reported as error in C API

[GitBox]

bputrycz opened a new issue #10715: Wrong params file is not reported as error 
in C API
URL: https://github.com/apache/incubator-mxnet/issues/10715
 
 
   Hi,
   
   I would like to report a possible problem in C API.
   
   I played with C API with a toy model (just adding two numbers) like:
   ```
   net = mx.gluon.nn.Dense(1, in_units=2)
   x = mx.sym.var('data')
   y = net(x)
   
   mod = mx.mod.Module(symbol=y, context=mx.cpu(), data_names = ['data'], 
label_names=[])
   mod.bind(data_shapes=[("data", (1,2))], for_training=False)
   mod.init_params(mx.init.Constant(1))
   mod.save_checkpoint('mx_add', 0)
   ```
   
   Then using code for prediction like:
   ```
   PredictorHandle pred_handler = NULL;
   
   const char* input_keys[] = { "data" };
   const mx_uint input_shape_indptr[] = { 0, 2 };
   const mx_uint input_shape_data[] = { 1, 2 };
   
   const mx_float input[] = {3, 5};
   mx_float output[1];
   
   int result;
   
   // In symbol/params_map we have appropriate files mapped into memory
   result = MXPredCreate(symbol_map->mem,
   params_map->mem,
   params_map->mapsize,
   1, /* dev_type: cpu */
   0, /* dev_id */
   1, /* num_input_nodes */
   input_keys,
   input_shape_indptr,
   input_shape_data,
   _handler);
   printf("%d, '%s'\n", result, MXGetLastError());
 
   result = MXPredSetInput(pred_handler, input_keys[0], input, 2);
   printf("%d, '%s'\n", result, MXGetLastError());
   
   result = MXPredForward(pred_handler);
   printf("%d, '%s'\n", result, MXGetLastError());
   
   result = MXPredGetOutput(pred_handler, 0, output, 1);
   printf("%d, '%s'\n", result, MXGetLastError());
   
   printf("sum: %f\n", output[0]);
   
   result = MXPredFree(pred_handler);
   printf("%d, '%s'\n", result, MXGetLastError());
   ```
   
   I was able to obtain, as expected, sum to be 8:
   ```
   0, ''
   0, ''
   0, ''
   0, ''
   sum: 8.00
   0, ''
   ```
   
   BUT: when I put into params file something wrong, like weight names with not 
matching names,
   then the prediction doesn’t report any error, and results with unspecified 
output:
   ```
   0, ''
   0, ''
   0, ''
   0, ''
   sum: 18980598159864999268117905408.00
   0, ''
   ```
   
   Shouldn’t predictor creation fail with some error about loading the params 
file?
   
   Bartosz
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on issue #10714: [MXNET-364] broadcast_add/sub between CSR and 1D dense vector

[GitBox]

haojin2 commented on issue #10714: [MXNET-364] broadcast_add/sub between CSR 
and 1D dense vector
URL: https://github.com/apache/incubator-mxnet/pull/10714#issuecomment-384824590
 
 
   @eric-haibin-lin 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 opened a new pull request #10714: [MXNET-364] broadcast_add/sub between CSR and 1D dense vector

[GitBox]

haojin2 opened a new pull request #10714: [MXNET-364] broadcast_add/sub between 
CSR and 1D dense vector
URL: https://github.com/apache/incubator-mxnet/pull/10714
 
 
   ## Description ##
   Same as title
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] The PR title starts with [MXNET-364]
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] Support broadcast_add/sub between csr matrices and 1D dense vector
   - [x] Corresponding unit tests
   
   ## Comments ##
   (Placeholder for benchmark results)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #10558: NNVM build failed in the newst mxnet version

[GitBox]

rahul003 commented on issue #10558: NNVM build failed in the newst mxnet version
URL: 
https://github.com/apache/incubator-mxnet/issues/10558#issuecomment-384824388
 
 
   Build with gcc-6.4 on Ubuntu16.04 works for me with cuda9.0. I'm not sure 
what changes with ubuntu17.04. Cuda9.1 should not affect this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] larroy commented on issue #10711: [MXNET-361] Fix cmake 3.11

[GitBox]

larroy commented on issue #10711: [MXNET-361] Fix cmake 3.11
URL: https://github.com/apache/incubator-mxnet/pull/10711#issuecomment-384823754
 
 
   LGTM, this is the right hack for the current CMake situation. It would be 
good if they could fix this in CMake, so far looks like there's a limitation in 
CMake when you want to build a dynamic library from an already compiled static 
library.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on a change in pull request #10713: [MXNET-363] fix race condition in sparse dot(csr.T, dense)

[GitBox]

haojin2 commented on a change in pull request #10713: [MXNET-363] fix race 
condition in sparse dot(csr.T, dense)
URL: https://github.com/apache/incubator-mxnet/pull/10713#discussion_r184562505
 
 

 ##
 File path: src/operator/tensor/dot-inl.cuh
 ##
 @@ -671,8 +630,60 @@ inline void DotCsrDnsDnsImpl(const OpContext& ctx,
   });
 }
 
+struct DotCsrTransDnsRspKernel {
+  /*!
+   * \brief
+   * \param tid  global thread id
+   * \param out  output rsp matrix data
+   * \param lookup_table lookup table from row in lhs to row in dst
+   * \param sorted_indices   csr matrix column indices in sorted order
+   * \param nnz  number of non-zeros in csr matrix
+   * \param original_idx original indices to the unsorted csr column 
indices
+   * \param rhs  dns rhs data
+   * \param val_arraycsr matrix data
+   * \param idx_arraycsr matrix row indices
+   * \param row_length   length of a row in the output rsp matrix
+   */
+  template
+  __device__ __forceinline__ static void Map(int thread_id,
+ DType* out,
+ const IType* lookup_table,
+ const IType* sorted_indices,
+ const nnvm::dim_t nnz,
+ const IType* original_idx,
+ const DType* rhs,
+ const DType* val_array,
+ const IType* idx_array,
+ const nnvm::dim_t row_length) {
+int tid = thread_id / row_length;
+const nnvm::dim_t offset = thread_id % row_length;
+if (tid == 0 || sorted_indices[tid - 1] != sorted_indices[tid]) {
+  DType acc = 0;
+  const IType src_row_idx = sorted_indices[tid];
+  const IType dst_row_idx = lookup_table[src_row_idx];
+  const IType out_offset = dst_row_idx * row_length + offset;
+  do {
+const IType idx = original_idx[tid];
+const DType val = val_array[idx];
+const DType col_idx = idx_array[idx];
+const IType rhs_offset = col_idx * row_length + offset;
+acc += rhs[rhs_offset] * val;
+tid++;
+  } while (tid < nnz && sorted_indices[tid - 1] == sorted_indices[tid]);
+  out[out_offset] = acc;
+}
+  }
+};
+
+// Returns integer log2(a) rounded up
+inline int log2i(size_t a) {
+  int k = 1;
+  while (a >>= 1) k++;
+  return k;
+}
+
 /*!
- * \brief GPU Impl of dot(csr, dns) = rsp and dot(csr.T, dns) = rsp
+ * \brief GPU Impl of dot(csr.T, dns) = rsp
 
 Review comment:
   Maybe we should change to dot(csr, dns, transpose_a=True) = rsp?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on a change in pull request #10713: [MXNET-363] fix race condition in sparse dot(csr.T, dense)

[GitBox]

haojin2 commented on a change in pull request #10713: [MXNET-363] fix race 
condition in sparse dot(csr.T, dense)
URL: https://github.com/apache/incubator-mxnet/pull/10713#discussion_r184562505
 
 

 ##
 File path: src/operator/tensor/dot-inl.cuh
 ##
 @@ -671,8 +630,60 @@ inline void DotCsrDnsDnsImpl(const OpContext& ctx,
   });
 }
 
+struct DotCsrTransDnsRspKernel {
+  /*!
+   * \brief
+   * \param tid  global thread id
+   * \param out  output rsp matrix data
+   * \param lookup_table lookup table from row in lhs to row in dst
+   * \param sorted_indices   csr matrix column indices in sorted order
+   * \param nnz  number of non-zeros in csr matrix
+   * \param original_idx original indices to the unsorted csr column 
indices
+   * \param rhs  dns rhs data
+   * \param val_arraycsr matrix data
+   * \param idx_arraycsr matrix row indices
+   * \param row_length   length of a row in the output rsp matrix
+   */
+  template
+  __device__ __forceinline__ static void Map(int thread_id,
+ DType* out,
+ const IType* lookup_table,
+ const IType* sorted_indices,
+ const nnvm::dim_t nnz,
+ const IType* original_idx,
+ const DType* rhs,
+ const DType* val_array,
+ const IType* idx_array,
+ const nnvm::dim_t row_length) {
+int tid = thread_id / row_length;
+const nnvm::dim_t offset = thread_id % row_length;
+if (tid == 0 || sorted_indices[tid - 1] != sorted_indices[tid]) {
+  DType acc = 0;
+  const IType src_row_idx = sorted_indices[tid];
+  const IType dst_row_idx = lookup_table[src_row_idx];
+  const IType out_offset = dst_row_idx * row_length + offset;
+  do {
+const IType idx = original_idx[tid];
+const DType val = val_array[idx];
+const DType col_idx = idx_array[idx];
+const IType rhs_offset = col_idx * row_length + offset;
+acc += rhs[rhs_offset] * val;
+tid++;
+  } while (tid < nnz && sorted_indices[tid - 1] == sorted_indices[tid]);
+  out[out_offset] = acc;
+}
+  }
+};
+
+// Returns integer log2(a) rounded up
+inline int log2i(size_t a) {
+  int k = 1;
+  while (a >>= 1) k++;
+  return k;
+}
+
 /*!
- * \brief GPU Impl of dot(csr, dns) = rsp and dot(csr.T, dns) = rsp
+ * \brief GPU Impl of dot(csr.T, dns) = rsp
 
 Review comment:
   Maybe we should change to dot(csr, dns, transpose_a=True) = rsp?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on a change in pull request #10713: [MXNET-363] fix race condition in sparse dot(csr.T, dense)

[GitBox]

haojin2 commented on a change in pull request #10713: [MXNET-363] fix race 
condition in sparse dot(csr.T, dense)
URL: https://github.com/apache/incubator-mxnet/pull/10713#discussion_r184562411
 
 

 ##
 File path: src/operator/tensor/dot-inl.cuh
 ##
 @@ -31,6 +31,10 @@
 #include "./sort_op.h"
 #include "./util/tensor_util-inl.h"
 #include "./util/tensor_util-inl.cuh"
+#include "./indexing_op.h"
+#include "./init_op.h"
+#include "./sort_op.h"
+
 
 Review comment:
   Extra includes?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin opened a new pull request #10713: [MXNET-363] fix race condition in sparse dot(csr.T, dense)

[GitBox]

eric-haibin-lin opened a new pull request #10713: [MXNET-363] fix race 
condition in sparse dot(csr.T, dense)
URL: https://github.com/apache/incubator-mxnet/pull/10713
 
 
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on issue #10591: handle inplace in mkldnn FallBackCompute

[GitBox]

zheng-da commented on issue #10591: handle inplace in mkldnn FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#issuecomment-384818577
 
 
   can you add a test for this?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on issue #10641: How to implement custom loss functions without label assignments (unsupervised) ?

[GitBox]

ThomasDelteil commented on issue #10641: How to implement custom loss functions 
without label assignments (unsupervised) ?
URL: 
https://github.com/apache/incubator-mxnet/issues/10641#issuecomment-384817210
 
 
   answered here 
https://discuss.mxnet.io/t/how-to-implement-custom-loss-functions-without-label-assignments-unsupervised/967


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10710: [WIP] Add cuda version check to skip building quantization ops for versions less than 8.0

[GitBox]

marcoabreu commented on issue #10710: [WIP] Add cuda version check to skip 
building quantization ops for versions less than 8.0
URL: https://github.com/apache/incubator-mxnet/pull/10710#issuecomment-384816481
 
 
   Please note this change is not verifiable by CI since we only support 
CUDA>=8.0. Please validate it manually


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10616: [MXNET-362] ensure same mkldnn engine is used for consistency

[GitBox]

ashokei commented on issue #10616: [MXNET-362] ensure same mkldnn engine is 
used for consistency
URL: https://github.com/apache/incubator-mxnet/pull/10616#issuecomment-384816294
 
 
   @marcoabreu https://issues.apache.org/jira/browse/MXNET-362


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham commented on issue #10702: danga zone added

[GitBox]

aaronmarkham commented on issue #10702: danga zone added
URL: https://github.com/apache/incubator-mxnet/pull/10702#issuecomment-384815930
 
 
   This was a test.  Was teaching a class for other writers and editors to 
contribute to MXNet.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #10704: fix rnn layer repr

[GitBox]

piiswrong commented on issue #10704: fix rnn layer repr
URL: https://github.com/apache/incubator-mxnet/pull/10704#issuecomment-384814620
 
 
   use //?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] 02/02: add unittest for gluon mkldnn ndarray slice computation

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git

commit f236687cbb146c792c576d8467ff45f643860f19
Author: Ashok Emani 
AuthorDate: Wed Apr 25 13:47:37 2018 -0700

add unittest for gluon mkldnn ndarray slice computation
---
 tests/python/mkl/test_mkldnn.py | 20 
 1 file changed, 20 insertions(+)

diff --git a/tests/python/mkl/test_mkldnn.py b/tests/python/mkl/test_mkldnn.py
index a4c9c45..5a621b0 100644
--- a/tests/python/mkl/test_mkldnn.py
+++ b/tests/python/mkl/test_mkldnn.py
@@ -22,6 +22,8 @@ MKL-DNN related test cases
 import logging
 import os
 from sys import platform
+import numpy as np
+from mxnet.test_utils import assert_almost_equal
 
 
 def test_mkldnn_install():
@@ -90,5 +92,23 @@ def test_mkldnn_model():
 assert 0, "test_mkldnn_model exception in bind and execution"
 
 
+def test_mkldnn_ndarray_slice():
+"""
+This test will trigger gluon computation on mkldnn with ndarray slice
+"""
+
+import mxnet as mx
+from mxnet import gluon
+ctx = mx.cpu()
+net = gluon.nn.HybridSequential()
+with net.name_scope():
+net.add(gluon.nn.Conv2D(channels=32, kernel_size=3, activation=None))
+net.collect_params().initialize(ctx=ctx)
+x = mx.nd.array(np.ones([32, 3, 224, 224]), ctx)
+y = net(x)
+
+# trigger computation on ndarray slice
+assert_almost_equal(y[0].asnumpy()[0,0,0], 0.3376348)
+
 if __name__ == '__main__':
 test_mkldnn_install()

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[incubator-mxnet] branch master updated (b0649f6 -> f236687)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git.


from b0649f6  fix multi context hybrid
 new f3f2733  handle NDArray slice properly for mkldnn layout
 new f236687  add unittest for gluon mkldnn ndarray slice computation

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 src/ndarray/ndarray.cc  |  9 -
 tests/python/mkl/test_mkldnn.py | 20 
 2 files changed, 28 insertions(+), 1 deletion(-)

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[incubator-mxnet] 01/02: handle NDArray slice properly for mkldnn layout

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git

commit f3f2733875b6968b048c09794e31e146cd09d506
Author: Ashok Emani 
AuthorDate: Thu Apr 19 14:07:46 2018 -0700

handle NDArray slice properly for mkldnn layout
---
 src/ndarray/ndarray.cc | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/src/ndarray/ndarray.cc b/src/ndarray/ndarray.cc
index 1ef7759..b428c2c 100644
--- a/src/ndarray/ndarray.cc
+++ b/src/ndarray/ndarray.cc
@@ -525,11 +525,18 @@ NDArray NDArray::Reorder2Default() const {
   if (format == ptr_->mkl_mem_->GetFormat())
 return *this;
 
-  NDArray ret(shape(), ctx(), false, dtype());
+  // create new ndarray from  mkldnn layout
+  mkldnn::memory::desc from_desc = ptr_->mkl_mem_->GetPrimitiveDesc().desc();
+  TShape tshape(from_desc.data.ndims);
+  for (int i = 0; i < from_desc.data.ndims; i++) tshape[i] = 
from_desc.data.dims[i];
+  NDArray ret(tshape, ctx(), false, dtype());
   mkldnn::memory::primitive_desc def_pd = 
ptr_->mkl_mem_->GetPrimitiveDesc(format);
   CHECK(ret.ptr_->shandle.size >= def_pd.get_size());
   mkldnn::memory def_mem(def_pd, ret.ptr_->shandle.dptr);
   ptr_->mkl_mem_->ReorderTo(_mem);
+  // reshape as needed
+  ret.shape_ = shape_;
+  ret.byte_offset_ = byte_offset_;
   return ret;
 }
 

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[GitHub] piiswrong closed pull request #10619: handle NDArray slice properly for mkldnn layout

[GitBox]

piiswrong closed pull request #10619: handle NDArray slice properly for mkldnn 
layout
URL: https://github.com/apache/incubator-mxnet/pull/10619
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/ndarray/ndarray.cc b/src/ndarray/ndarray.cc
index 1ef7759b4da..b428c2cbefb 100644
--- a/src/ndarray/ndarray.cc
+++ b/src/ndarray/ndarray.cc
@@ -525,11 +525,18 @@ NDArray NDArray::Reorder2Default() const {
   if (format == ptr_->mkl_mem_->GetFormat())
 return *this;
 
-  NDArray ret(shape(), ctx(), false, dtype());
+  // create new ndarray from  mkldnn layout
+  mkldnn::memory::desc from_desc = ptr_->mkl_mem_->GetPrimitiveDesc().desc();
+  TShape tshape(from_desc.data.ndims);
+  for (int i = 0; i < from_desc.data.ndims; i++) tshape[i] = 
from_desc.data.dims[i];
+  NDArray ret(tshape, ctx(), false, dtype());
   mkldnn::memory::primitive_desc def_pd = 
ptr_->mkl_mem_->GetPrimitiveDesc(format);
   CHECK(ret.ptr_->shandle.size >= def_pd.get_size());
   mkldnn::memory def_mem(def_pd, ret.ptr_->shandle.dptr);
   ptr_->mkl_mem_->ReorderTo(_mem);
+  // reshape as needed
+  ret.shape_ = shape_;
+  ret.byte_offset_ = byte_offset_;
   return ret;
 }
 
diff --git a/tests/python/mkl/test_mkldnn.py b/tests/python/mkl/test_mkldnn.py
index a4c9c4557a3..5a621b0e26e 100644
--- a/tests/python/mkl/test_mkldnn.py
+++ b/tests/python/mkl/test_mkldnn.py
@@ -22,6 +22,8 @@
 import logging
 import os
 from sys import platform
+import numpy as np
+from mxnet.test_utils import assert_almost_equal
 
 
 def test_mkldnn_install():
@@ -90,5 +92,23 @@ def get_tensors(args, shapes, ctx):
 assert 0, "test_mkldnn_model exception in bind and execution"
 
 
+def test_mkldnn_ndarray_slice():
+"""
+This test will trigger gluon computation on mkldnn with ndarray slice
+"""
+
+import mxnet as mx
+from mxnet import gluon
+ctx = mx.cpu()
+net = gluon.nn.HybridSequential()
+with net.name_scope():
+net.add(gluon.nn.Conv2D(channels=32, kernel_size=3, activation=None))
+net.collect_params().initialize(ctx=ctx)
+x = mx.nd.array(np.ones([32, 3, 224, 224]), ctx)
+y = net(x)
+
+# trigger computation on ndarray slice
+assert_almost_equal(y[0].asnumpy()[0,0,0], 0.3376348)
+
 if __name__ == '__main__':
 test_mkldnn_install()


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KellenSunderland commented on a change in pull request #10711: [MXNET-361] Fix cmake 3.11

[GitBox]

KellenSunderland commented on a change in pull request #10711: [MXNET-361] Fix 
cmake 3.11
URL: https://github.com/apache/incubator-mxnet/pull/10711#discussion_r184552570
 
 

 ##
 File path: CMakeLists.txt
 ##
 @@ -584,9 +584,12 @@ endif()
 
 set(MXNET_INSTALL_TARGETS mxnet)
 if(UNIX)
+  # Create dummty file since we want an empty shared library before linking
 
 Review comment:
   dummty -> dummy


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10708: CMake 3.11 breaks on UNIX

[GitBox]

marcoabreu commented on issue #10708: CMake 3.11 breaks on UNIX
URL: 
https://github.com/apache/incubator-mxnet/issues/10708#issuecomment-384812431
 
 
   Note: This issue was not caught by our current CI since Cmake 3.11 has just 
been released and our docker cache has not been purged yet. Additionally, this 
version has no binary distribution on Ubuntu16.04 yet, but our cross-compile 
environment for ARMv6 is already using this version and thus failing the build.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rahul003 commented on issue #10699: Is the "gradient compression" only suitable for range_pull?

[GitBox]

rahul003 commented on issue #10699: Is the "gradient compression" only suitable 
for range_pull?
URL: 
https://github.com/apache/incubator-mxnet/issues/10699#issuecomment-384811000
 
 
   Could you clarify what you are referring to by hash_push or range_push?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] David-Levinthal commented on issue #10505: Profiler profiler shoudl collect call durations not just timestamps

[GitBox]

David-Levinthal commented on issue #10505: Profiler  profiler shoudl collect 
call durations not just timestamps
URL: 
https://github.com/apache/incubator-mxnet/issues/10505#issuecomment-384808874
 
 
   the bigger problem then seems to be creating multiple files (ex one per
   minibatch) and getting the process records (ph="M") and opening and closing
   brackets etc
   but I should probably submit a separate report for that..and let this one
   get closed
   
   
   On Thu, Apr 26, 2018 at 2:54 PM, Abhishek Tiwari 
   wrote:
   
   > Yes they are always sequential with the order: "E" event following "B"
   > event. This can be seen in the code in file profiler.cc.
   >
   > The code prints out device information ie. an entry for each cpu thread,
   > gpu thread and one for pinned memory. Then, it prints, for each Operator
   > Executor Stat in each Profiler Stat, first a B event then a E event.
   >
   > I think the authors saved and printed timestamps instead of duration
   > because duration can be derived from timestamps thus timestamps contain
   > more information for debugging purposes than duration alone.
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu opened a new pull request #10712: Fix cmake 3.11

[GitBox]

marcoabreu opened a new pull request #10712: Fix cmake 3.11
URL: https://github.com/apache/incubator-mxnet/pull/10712
 
 
   See https://github.com/apache/incubator-mxnet/issues/10708 for details


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu opened a new pull request #10711: Fix for cmake 3.11

[GitBox]

marcoabreu opened a new pull request #10711: Fix for cmake 3.11
URL: https://github.com/apache/incubator-mxnet/pull/10711
 
 
   See https://github.com/apache/incubator-mxnet/issues/10708 for details


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] reminisce opened a new pull request #10710: [WIP] Add cuda version check to skip building quantization ops for versions less than 8.0

[GitBox]

reminisce opened a new pull request #10710: [WIP] Add cuda version check to 
skip building quantization ops for versions less than 8.0
URL: https://github.com/apache/incubator-mxnet/pull/10710
 
 
   
   ## Description ##
   Build would fail if users installed cuda prior to version 8.0. Added cuda 
version check to prevent build failure.
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin opened a new issue #10709: [SPARSE] undeterministic result of sparse dot(csr, dense, transpose_a=True)

[GitBox]

eric-haibin-lin opened a new issue #10709: [SPARSE] undeterministic result of 
sparse dot(csr, dense, transpose_a=True)
URL: https://github.com/apache/incubator-mxnet/issues/10709
 
 
   The following test failed because of the limited precision of fp32 and the 
usage of atomicAdd in sparse dot implementation: 
   ```
   def check_dot_determinism(lhs_stype, rhs_stype, lhs_density, 
rhs_density, transpose_a, transpose_b):
   lhs_shape = (2, 200)
   rhs_shape = (2, 200)
   lhs = rand_ndarray(lhs_shape, lhs_stype, density=lhs_density)
   rhs = rand_ndarray(rhs_shape, rhs_stype, density=rhs_density)
   res1 = mx.nd.sparse.dot(lhs, rhs, transpose_a=transpose_a, 
transpose_b=transpose_b)
   res2 = mx.nd.sparse.dot(lhs, rhs, transpose_a=transpose_a, 
transpose_b=transpose_b)
   
   assert_almost_equal(res1.asnumpy(), res2.asnumpy(), rtol=0.0, 
atol=0.0)
   check_dot_determinism('csr', 'default', 0.5, 1.0, True, False)
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu opened a new issue #10708: CMake 3.11 breaks on UNIX

[GitBox]

marcoabreu opened a new issue #10708: CMake 3.11 breaks on UNIX
URL: https://github.com/apache/incubator-mxnet/issues/10708
 
 
   With the last release of CMake 3.11, our CMakeLists is invalid and breaks on 
all UNIX-based systems:
   
   ```
   cmake -DCMAKE_TOOLCHAIN_FILE=/usr/arm-linux-gnueabihf/Toolchain.cmake 
-DUSE_CUDA=OFF -DUSE_OPENCV=OFF -DUSE_OPENMP=OFF -DUSE_SIGNAL_HANDLER=ON 
-DCMAKE_BUILD_TYPE=Release -DUSE_MKL_IF_AVAILABLE=OFF -DUSE_LAPACK=OFF 
-DBUILD_CPP_EXAMPLES=OFF -Dmxnet_LINKER_LIBS=-lgfortran -G Ninja /work/mxnet
   
   -- The C compiler identification is GNU 4.8.3
   
   -- The CXX compiler identification is GNU 4.8.3
   
   -- Check for working C compiler: /usr/bin/arm-linux-gnueabihf-gcc
   
   -- Check for working C compiler: /usr/bin/arm-linux-gnueabihf-gcc -- works
   
   -- Detecting C compiler ABI info
   
   -- Detecting C compiler ABI info - done
   
   -- Detecting C compile features
   
   -- Detecting C compile features - done
   
   -- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++
   
   -- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++ -- works
   
   -- Detecting CXX compiler ABI info
   
   -- Detecting CXX compiler ABI info - done
   
   -- Detecting CXX compile features
   
   -- Detecting CXX compile features - done
   
   -- Performing Test SUPPORT_CXX11
   
   -- Performing Test SUPPORT_CXX11 - Success
   
   -- Performing Test SUPPORT_CXX0X
   
   -- Performing Test SUPPORT_CXX0X - Success
   
   -- Performing Test SUPPORT_MSSE2
   
   -- Performing Test SUPPORT_MSSE2 - Failed
   
   -- Found OpenBLAS libraries: /usr/arm-linux-gnueabihf/lib/libopenblas.so
   
   -- Found OpenBLAS include: /usr/arm-linux-gnueabihf/include
   
   -- Performing Test COMPILER_SUPPORT_MF16C
   
   -- Performing Test COMPILER_SUPPORT_MF16C - Failed
   
   -- Could NOT find Gperftools (missing: GPERFTOOLS_LIBRARIES 
GPERFTOOLS_INCLUDE_DIR) 
   
   -- Found PkgConfig: /usr/bin/pkg-config (found version "0.28") 
   
   -- Could NOT find Jemalloc (missing: JEMALLOC_LIBRARY JEMALLOC_INCLUDE_DIR) 
   
   -- OpenCV Disabled
   
   -- Could NOT find Jemalloc (missing: JEMALLOC_LIBRARY JEMALLOC_INCLUDE_DIR) 
   
   -- Found PythonInterp: /usr/bin/python (found version "2.7.9") 
   
   -- Looking for pthread.h
   
   -- Looking for pthread.h - found
   
   -- Looking for pthread_create
   
   -- Looking for pthread_create - not found
   
   -- Looking for pthread_create in pthreads
   
   -- Looking for pthread_create in pthreads - not found
   
   -- Looking for pthread_create in pthread
   
   -- Looking for pthread_create in pthread - found
   
   -- Found Threads: TRUE  
   
   -- Found GTest: gtest  
   
   -- Configuring done
   
   CMake Error at CMakeLists.txt:589 (add_library):
   
 No SOURCES given to target: mxnet
   ```
   
   This is caused by the following code: 
   ```
   set(MXNET_INSTALL_TARGETS mxnet)
   if(UNIX)
 list(APPEND MXNET_INSTALL_TARGETS mxnet_static)
 add_library(mxnet_static STATIC ${SOURCE})
 add_library(mxnet SHARED )
 target_link_libraries(mxnet PRIVATE ${BEGIN_WHOLE_ARCHIVE} 
$ ${END_WHOLE_ARCHIVE})
 target_link_libraries(mxnet PRIVATE mxnet_static)
 set_target_properties(mxnet_static PROPERTIES OUTPUT_NAME mxnet)
   else()
   ```
   
   Reason here is that ```add_library``` now requires at least one source file. 
Previously, this was indicated by a warning which has now been upgraded to an 
error.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] olympian94 commented on issue #10505: Profiler profiler shoudl collect call durations not just timestamps

[GitBox]

olympian94 commented on issue #10505: Profiler  profiler shoudl collect call 
durations not just timestamps
URL: 
https://github.com/apache/incubator-mxnet/issues/10505#issuecomment-384800848
 
 
   Yes they are always sequential with the order: "E" event following "B" 
event. This can be seen in the code in file profiler.cc.
   
   The code prints out device information ie. an entry for each cpu thread, gpu 
thread and one for pinned memory. Then, it prints, for each Operator Executor 
Stat in each Profiler Stat, first a B event then a E event.
   
   I think the authors saved and printed timestamps instead of duration because 
duration can be derived from timestamps thus timestamps contain more 
information for debugging purposes than duration alone.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 opened a new pull request #10707: [MXNET-357] New Scala API Design (NDArray)

[GitBox]

lanking520 opened a new pull request #10707: [MXNET-357] New Scala API Design 
(NDArray)
URL: https://github.com/apache/incubator-mxnet/pull/10707
 
 
   ## Description ##
   See [full design 
document](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Scala+API+Usability+Improvement)
   @nswamy @yzhliu 
   This PR is the Addition for new NDArray functions of Scala API
   ## Checklist ##
   ### Essentials ###
   - [x] Use QuasiQuote to replace original API Implementation (Reduce lines)
   - [x] MakeAtomicFunction change to support String type parameter type in
   - [x] User New namespace for New API (temporarily New)
   - [x] Default args using None to pass in as default
   - [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] David-Levinthal commented on issue #10505: Profiler profiler shoudl collect call durations not just timestamps

[GitBox]

David-Levinthal commented on issue #10505: Profiler  profiler shoudl collect 
call durations not just timestamps
URL: 
https://github.com/apache/incubator-mxnet/issues/10505#issuecomment-384798286
 
 
   The problem is matching up records or are the B & E records always
   sequential?
   It would be easier to process if only one record were written out with the
   duration
   which is what Tensorflow's timeline does.
   
   
   On Thu, Apr 26, 2018 at 1:51 PM, Abhishek Tiwari 
   wrote:
   
   > My understanding is that B and E signify begin and end event mark
   > respectively so you can get the duration in microseconds by subtracting the
   > timestamps
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on issue #10670: enable jenkins for mkldnn unittest

[GitBox]

zheng-da commented on issue #10670: enable jenkins for mkldnn unittest
URL: https://github.com/apache/incubator-mxnet/pull/10670#issuecomment-384795201
 
 
   This PR isn't needed any more. The test in the PR has been merged in another 
PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] larroy commented on issue #10297: [MXNET-244][WIP Don't merge] Fixes for cross compilation in ARM

[GitBox]

larroy commented on issue #10297: [MXNET-244][WIP Don't merge] Fixes for cross 
compilation in ARM
URL: https://github.com/apache/incubator-mxnet/pull/10297#issuecomment-384786576
 
 
   Hi, sorry been in holiday and busy with the machine learning conference. We 
will retake this in the following weeks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] olympian94 commented on issue #10505: Profiler profiler shoudl collect call durations not just timestamps

[GitBox]

olympian94 commented on issue #10505: Profiler  profiler shoudl collect call 
durations not just timestamps
URL: 
https://github.com/apache/incubator-mxnet/issues/10505#issuecomment-384784776
 
 
   My understanding is that B and E signify begin and end event mark 
respectively so you can get the duration in microseconds by subtracting the 
timestamps


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da opened a new pull request #10706: invalidate MKLDNN memory for reused NDArrays.

[GitBox]

zheng-da opened a new pull request #10706: invalidate MKLDNN memory for reused 
NDArrays.
URL: https://github.com/apache/incubator-mxnet/pull/10706
 
 
   ## Description ##
   we need to invalidate the MKLDNN arrays when they are reused as output 
arrays. We have done it in the symbolic executor. We need to do the same thing 
in the imperative execution.
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on issue #10624: [MXNET-351] Fix a bug in the MKLDNN integration.

[GitBox]

zheng-da commented on issue #10624: [MXNET-351] Fix a bug in the MKLDNN 
integration.
URL: https://github.com/apache/incubator-mxnet/pull/10624#issuecomment-384783346
 
 
   `build/3rdparty/mkldnn/src/libmkldnn.so` and 
`build/3rdparty/mkldnn/src/libmkldnn.so.0` are essentially the same file. 
That's why they have the same hash.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10591: handle inplace in mkldnn FallBackCompute

[GitBox]

ashokei commented on issue #10591: handle inplace in mkldnn FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#issuecomment-384781152
 
 
   @zheng-da added your feedback comments, @piiswrong can you please merge if 
ok. thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on a change in pull request #10616: ensure same mkldnn engine is used for consistency

[GitBox]

ashokei commented on a change in pull request #10616: ensure same mkldnn engine 
is used for consistency
URL: https://github.com/apache/incubator-mxnet/pull/10616#discussion_r184523143
 
 

 ##
 File path: tests/python/mkl/test_mkldnn.py
 ##
 @@ -89,6 +91,33 @@ def get_tensors(args, shapes, ctx):
 except:  # pylint: disable=bare-except
 assert 0, "test_mkldnn_model exception in bind and execution"
 
+def test_mkldnn_engine_threading():
+"""
+This test will trigger mkldnn engine on different thread of execution
+"""
+
+import mxnet as mx
+from mxnet import gluon, nd
+
+net = gluon.nn.HybridSequential()
+with net.name_scope():
+net.add(gluon.nn.Conv2D(channels=32, kernel_size=3, activation=None))
+net.collect_params().initialize(ctx=mx.cpu())
+
+val_data = gluon.data.DataLoader(
+gluon.data.vision.CIFAR10(train=False),
+batch_size=32, shuffle=False, num_workers=1)
 
 Review comment:
   Gluon DataLoader allows us to create a new thread as it iterates over data 
batch. Added comments to PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on a change in pull request #10616: ensure same mkldnn engine is used for consistency

[GitBox]

ashokei commented on a change in pull request #10616: ensure same mkldnn engine 
is used for consistency
URL: https://github.com/apache/incubator-mxnet/pull/10616#discussion_r184280082
 
 

 ##
 File path: tests/python/mkl/test_mkldnn.py
 ##
 @@ -89,6 +91,33 @@ def get_tensors(args, shapes, ctx):
 except:  # pylint: disable=bare-except
 assert 0, "test_mkldnn_model exception in bind and execution"
 
+def test_mkldnn_engine_threading():
+"""
+This test will trigger mkldnn engine on different thread of execution
+"""
+
+import mxnet as mx
+from mxnet import gluon, nd
+
+net = gluon.nn.HybridSequential()
+with net.name_scope():
+net.add(gluon.nn.Conv2D(channels=32, kernel_size=3, activation=None))
+net.collect_params().initialize(ctx=mx.cpu())
+
+val_data = gluon.data.DataLoader(
+gluon.data.vision.CIFAR10(train=False),
+batch_size=32, shuffle=False, num_workers=1)
 
 Review comment:
   actually, the dataloader is the one that triggers different thread context, 
i tried without a dataloader (or even if i pass num_workers = 0 to above 
dataloader) it runs on same thread, so bug wont happen. Gluon DataLoader allows 
us to create a new thread as it iterates over data batch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on a change in pull request #10616: ensure same mkldnn engine is used for consistency

[GitBox]

zheng-da commented on a change in pull request #10616: ensure same mkldnn 
engine is used for consistency
URL: https://github.com/apache/incubator-mxnet/pull/10616#discussion_r184518463
 
 

 ##
 File path: tests/python/mkl/test_mkldnn.py
 ##
 @@ -89,6 +91,33 @@ def get_tensors(args, shapes, ctx):
 except:  # pylint: disable=bare-except
 assert 0, "test_mkldnn_model exception in bind and execution"
 
+def test_mkldnn_engine_threading():
+"""
+This test will trigger mkldnn engine on different thread of execution
+"""
+
+import mxnet as mx
+from mxnet import gluon, nd
+
+net = gluon.nn.HybridSequential()
+with net.name_scope():
+net.add(gluon.nn.Conv2D(channels=32, kernel_size=3, activation=None))
+net.collect_params().initialize(ctx=mx.cpu())
+
+val_data = gluon.data.DataLoader(
+gluon.data.vision.CIFAR10(train=False),
+batch_size=32, shuffle=False, num_workers=1)
 
 Review comment:
   yes, please provide comments why we need data loader here. Thanks


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] redpine42 commented on issue #10705: Problems building and running mxnet on machines with no AVX2 support

[GitBox]

redpine42 commented on issue #10705: Problems building and running mxnet on 
machines with no AVX2 support
URL: 
https://github.com/apache/incubator-mxnet/issues/10705#issuecomment-384774783
 
 
   Build error one E5-2690
   
   ```
   Step 6/8 : RUN bash && cd /incubator-mxnet/python && pip install -e .
---> Running in 8b11a421dfef
   Obtaining file:///incubator-mxnet/python
   Collecting numpy<=1.15.0,>=1.8.2 (from mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/76/4d/418dda252cf92bad00ab82d6b 

   
2a856e7843b47a5c2f084aed34b14b67d64/numpy-1.14.2-cp27-cp27mu-manylinux1_x86_64.whl
 (1 
   
2.1MB)
   Collecting requests<2.19.0,>=2.18.4 (from mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d 

   
9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)
   Collecting graphviz<0.9.0,>=0.8.1 (from mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/05/e4/8fcc76823534d47f079c0ff1b 

   
3d8b57784e8fba63ceb1ded32c9f4dd993c/graphviz-0.8.2-py2.py3-none-any.whl
   Collecting certifi>=2017.4.17 (from requests<2.19.0,>=2.18.4->mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b 

   
65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl 
(150kB)
   Collecting chardet<3.1.0,>=3.0.2 (from 
requests<2.19.0,>=2.18.4->mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb 

   
1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
   Collecting idna<2.7,>=2.5 (from requests<2.19.0,>=2.18.4->mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b99 

   
2838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
   Collecting urllib3<1.23,>=1.21.1 (from 
requests<2.19.0,>=2.18.4->mxnet==1.2.0)
 Downloading 
https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223 

   
e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)
   Installing collected packages: numpy, certifi, chardet, idna, urllib3, 
requests, grap  

  hviz, mxnet
 Running setup.py develop for mxnet
   Complete output from command /usr/bin/python -c "import setuptools, 
tokenize;__fi   

 le__='/incubator-mxnet/python/setup.py';f=getattr(tokenize, 'open', 
open)(__file__);c   

 ode=f.read().replace('\r\n', '\n');f.close();exec(compile(code, 
__file__, 'exec'))" d   

 evelop --no-deps:
   
   
   Command "/usr/bin/python -c "import setuptools, 
tokenize;__file__='/incubator-mxnet/p   

 ython/setup.py';f=getattr(tokenize, 'open', 
open)(__file__);code=f.read().replace('\r   

 \n', '\n');f.close();exec(compile(code, 
__file__, 'exec'))" develop --no-deps" failed   
  

[GitHub] marcoabreu commented on a change in pull request #10616: ensure same mkldnn engine is used for consistency

[GitBox]

marcoabreu commented on a change in pull request #10616: ensure same mkldnn 
engine is used for consistency
URL: https://github.com/apache/incubator-mxnet/pull/10616#discussion_r184512853
 
 

 ##
 File path: tests/python/mkl/test_mkldnn.py
 ##
 @@ -89,6 +91,33 @@ def get_tensors(args, shapes, ctx):
 except:  # pylint: disable=bare-except
 assert 0, "test_mkldnn_model exception in bind and execution"
 
+def test_mkldnn_engine_threading():
+"""
+This test will trigger mkldnn engine on different thread of execution
+"""
+
+import mxnet as mx
+from mxnet import gluon, nd
+
+net = gluon.nn.HybridSequential()
+with net.name_scope():
+net.add(gluon.nn.Conv2D(channels=32, kernel_size=3, activation=None))
+net.collect_params().initialize(ctx=mx.cpu())
+
+val_data = gluon.data.DataLoader(
+gluon.data.vision.CIFAR10(train=False),
+batch_size=32, shuffle=False, num_workers=1)
 
 Review comment:
   Thanks a lot for this test! Could you please elaborate the exact behaviour 
of this unit test in the test with a block comment. I agree with Da that at the 
moment, it's hard to grasp the exact problem from reading the code. For me it's 
hard to understand when different threads are getting started and what the 
exact issue is.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] redpine42 opened a new issue #10705: Problems building and running mxnet on machines with no AVX2 support

[GitBox]

redpine42 opened a new issue #10705: Problems building and running mxnet on 
machines with no AVX2 support
URL: https://github.com/apache/incubator-mxnet/issues/10705
 
 
   I'm having problems building and running mxnet from this repo on machines 
that don't support AVX2.  
   
   I can build and run on machines support AVX2.
   I'm following these instructions for a docker build and I've configured for 
distributed processing ( USE_DIST_KVSTORE=1).  
https://mxnet.incubator.apache.org/install/index.html
   
   Below is the dockerfile instructions.
   
   ```
   FROM ubuntu:16.04
   
   RUN apt-get update && \
   apt-get install -y wget build-essential git libopenblas-dev 
liblapack-dev \
   libopencv-dev
   
   RUN cd / && git clone --recursive https://github.com/apache/incubator-mxnet 
&& cd incubator-mxnet && \
   make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_DIST_KVSTORE=1 
DEBUG=1
   
   RUN apt-get install -y python-dev python-setuptools python-pip libgfortran3
   
   RUN python -m pip install --upgrade pip && \
   pip install --upgrade setuptools
   
   RUN bash && cd /incubator-mxnet/python && pip install -e .
   
   RUN mkdir /root/.ssh && chmod 700 /root/.ssh
   ENV PYTHONPATH=/incubator-mxnet/python
   ```
   I can build and run on  Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz.
   
   This fails a run on an E5-2690.  I also can not build on this machine.
   
   Both machinces are running on Centos 7.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10629: [MXNET-343]fix Mkldnn with msvc

[GitBox]

marcoabreu commented on issue #10629: [MXNET-343]fix Mkldnn with msvc
URL: https://github.com/apache/incubator-mxnet/pull/10629#issuecomment-384767150
 
 
   Would be good if somebody could review the cmake changes. @cjolivier01 
@eric-haibin-lin 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #10629: [MXNET-343]fix Mkldnn with msvc

[GitBox]

marcoabreu commented on a change in pull request #10629: [MXNET-343]fix Mkldnn 
with msvc
URL: https://github.com/apache/incubator-mxnet/pull/10629#discussion_r184509001
 
 

 ##
 File path: cmake/FirstClassLangCuda.cmake
 ##
 @@ -126,7 +126,7 @@ endif ()
 function(mshadow_select_nvcc_arch_flags out_variable)
 
   set(CUDA_ARCH_LIST "Auto" CACHE STRING "Select target NVIDIA GPU 
achitecture.")
-  set_property( CACHE CUDA_ARCH_LIST PROPERTY STRINGS "" "All" "Common" 
${CUDA_KNOWN_GPU_ARCHITECTURES} )
+  set_property( CACHE CUDA_ARCH_LIST PROPERTY STRINGS "" "Auto" "All" "Common" 
${CUDA_KNOWN_GPU_ARCHITECTURES} )
 
 Review comment:
   This is a default behaviour change, right?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10670: enable jenkins for mkldnn unittest

[GitBox]

marcoabreu commented on issue #10670: enable jenkins for mkldnn unittest
URL: https://github.com/apache/incubator-mxnet/pull/10670#issuecomment-384766207
 
 
   @zheng-da @ashokei please ping me if there's any action requested from me. 
Since this PR is closed, I'll wait for further notice


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10668: [feature request] Jenkins build restartable from comments in the PR

[GitBox]

marcoabreu commented on issue #10668: [feature request] Jenkins build 
restartable from comments in the PR
URL: 
https://github.com/apache/incubator-mxnet/issues/10668#issuecomment-384765311
 
 
   Looks good, I'll definitely look into this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #8797: fix errors for reinterpret_cast usage

[GitBox]

marcoabreu commented on issue #8797: fix errors for reinterpret_cast usage
URL: https://github.com/apache/incubator-mxnet/pull/8797#issuecomment-384764755
 
 
   @xinghedyc could you please rebase? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10624: [MXNET-351] Fix a bug in the MKLDNN integration.

[GitBox]

marcoabreu commented on issue #10624: [MXNET-351] Fix a bug in the MKLDNN 
integration.
URL: https://github.com/apache/incubator-mxnet/pull/10624#issuecomment-384764323
 
 
   I agree, this loods strange. But the filehashes of these files look odd as 
well - could it be possible they're empty or don't exist? Would be good if you 
could dive a bit deeper there.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: fix multi context hybrid

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new b0649f6  fix multi context hybrid
b0649f6 is described below

commit b0649f61f513a78d6f45888b69f4c0798a8cde0c
Author: Junyuan Xie 
AuthorDate: Wed Apr 25 12:36:10 2018 -0700

fix multi context hybrid
---
 python/mxnet/_ctypes/ndarray.py| 2 +-
 src/imperative/cached_op.cc| 3 ++-
 tests/python/unittest/test_gluon.py| 9 +
 tests/python/unittest/test_operator.py | 9 +
 4 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/python/mxnet/_ctypes/ndarray.py b/python/mxnet/_ctypes/ndarray.py
index 191985e..d2cae0c 100644
--- a/python/mxnet/_ctypes/ndarray.py
+++ b/python/mxnet/_ctypes/ndarray.py
@@ -124,7 +124,7 @@ class CachedOp(object):
 c_str_array([str(val) for _, val in flags]),
 len(inputs),
 c_str_array(inputs),
-len(params),
+len(param_names),
 c_str_array(param_names),
 c_handle_array(param_arrays),
 ctypes.byref(self.handle)))
diff --git a/src/imperative/cached_op.cc b/src/imperative/cached_op.cc
index 10a8fc0..140b5a5 100644
--- a/src/imperative/cached_op.cc
+++ b/src/imperative/cached_op.cc
@@ -89,7 +89,8 @@ Imperative::CachedOp::CachedOp(
 }
 
 CHECK_EQ(arg_name_to_id.size(), arg_names.size())
-<< "Expecting " << arg_name_to_id.size() << "inputs, given " << 
arg_names.size();
+<< "CachedOp expects " << arg_name_to_id.size()
+<< " inputs, given " << arg_names.size();
 
 for (const auto& name : arg_names) {
   auto iter = arg_name_to_id.find(name);
diff --git a/tests/python/unittest/test_gluon.py 
b/tests/python/unittest/test_gluon.py
index abb27de..456bcf4 100644
--- a/tests/python/unittest/test_gluon.py
+++ b/tests/python/unittest/test_gluon.py
@@ -25,6 +25,8 @@ from nose.tools import raises
 from copy import deepcopy
 import warnings
 import json
+import unittest
+
 
 
 @with_seed()
@@ -967,6 +969,13 @@ def test_save_load():
 net.load_params('test.params')
 
 
+@unittest.skip("Fails due to an error in mkldnn implementation unrelated to 
what we want to test here.")
+def test_hybrid_multi_context():
+net = mx.gluon.model_zoo.vision.get_resnet(1, 18)
+net.initialize(ctx=[mx.cpu(0), mx.cpu(1)])
+net.hybridize()
+net(mx.nd.zeros((1, 3, 32, 32), ctx=mx.cpu(0))).asnumpy()
+
 
 if __name__ == '__main__':
 import nose
diff --git a/tests/python/unittest/test_operator.py 
b/tests/python/unittest/test_operator.py
index 89557d1..6e5c908 100644
--- a/tests/python/unittest/test_operator.py
+++ b/tests/python/unittest/test_operator.py
@@ -601,7 +601,7 @@ def test_hard_sigmoid():
 for dtype in [np.float16, np.float32, np.float64]:
 if dtype is np.float16:
 rtol = 1e-2
-atol = 1e-4
+atol = 1e-3
 else:
 rtol = 1e-3
 atol = 1e-5
@@ -611,9 +611,10 @@ def test_hard_sigmoid():
 xa[xa == -2.5] = xa[xa == -2.5] - 1e-2
 ya = fhardsigmoid(xa)
 grad_xa = fhardsigmoid_grad(xa, np.ones(shape))
-check_numeric_gradient(y, [xa], numeric_eps=1e-3, rtol=rtol, atol=atol)
-check_symbolic_forward(y, [xa], [ya], rtol=rtol, atol=atol)
-check_symbolic_backward(y, [xa], [np.ones(shape)], [grad_xa], 
rtol=rtol, atol=atol)
+if dtype is not np.float16:
+check_numeric_gradient(y, [xa], numeric_eps=1e-3, rtol=rtol, 
atol=atol, dtype=dtype)
+check_symbolic_forward(y, [xa], [ya], rtol=rtol, atol=atol, 
dtype=dtype)
+check_symbolic_backward(y, [xa], [np.ones(shape)], [grad_xa], 
rtol=rtol, atol=atol, dtype=dtype)
 
 @with_seed()
 def test_softsign():

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[GitHub] piiswrong closed pull request #10687: Fix multi context hybrid

[GitBox]

piiswrong closed pull request #10687: Fix multi context hybrid
URL: https://github.com/apache/incubator-mxnet/pull/10687
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/mxnet/_ctypes/ndarray.py b/python/mxnet/_ctypes/ndarray.py
index 191985edc5c..d2cae0c45aa 100644
--- a/python/mxnet/_ctypes/ndarray.py
+++ b/python/mxnet/_ctypes/ndarray.py
@@ -124,7 +124,7 @@ def __init__(self, sym, flags=(), inputs=None, params=None):
 c_str_array([str(val) for _, val in flags]),
 len(inputs),
 c_str_array(inputs),
-len(params),
+len(param_names),
 c_str_array(param_names),
 c_handle_array(param_arrays),
 ctypes.byref(self.handle)))
diff --git a/src/imperative/cached_op.cc b/src/imperative/cached_op.cc
index 10a8fc0c6ba..140b5a5d81e 100644
--- a/src/imperative/cached_op.cc
+++ b/src/imperative/cached_op.cc
@@ -89,7 +89,8 @@ Imperative::CachedOp::CachedOp(
 }
 
 CHECK_EQ(arg_name_to_id.size(), arg_names.size())
-<< "Expecting " << arg_name_to_id.size() << "inputs, given " << 
arg_names.size();
+<< "CachedOp expects " << arg_name_to_id.size()
+<< " inputs, given " << arg_names.size();
 
 for (const auto& name : arg_names) {
   auto iter = arg_name_to_id.find(name);
diff --git a/tests/python/unittest/test_gluon.py 
b/tests/python/unittest/test_gluon.py
index 0a5bda831d9..dd4da756ee4 100644
--- a/tests/python/unittest/test_gluon.py
+++ b/tests/python/unittest/test_gluon.py
@@ -25,6 +25,8 @@
 from copy import deepcopy
 import warnings
 import json
+import unittest
+
 
 
 @with_seed()
@@ -967,6 +969,13 @@ def test_save_load():
 net.load_params('test.params')
 
 
+@unittest.skip("Fails due to an error in mkldnn implementation unrelated to 
what we want to test here.")
+def test_hybrid_multi_context():
+net = mx.gluon.model_zoo.vision.get_resnet(1, 18)
+net.initialize(ctx=[mx.cpu(0), mx.cpu(1)])
+net.hybridize()
+net(mx.nd.zeros((1, 3, 32, 32), ctx=mx.cpu(0))).asnumpy()
+
 
 if __name__ == '__main__':
 import nose
diff --git a/tests/python/unittest/test_operator.py 
b/tests/python/unittest/test_operator.py
index 89557d1c497..6e5c9081c31 100644
--- a/tests/python/unittest/test_operator.py
+++ b/tests/python/unittest/test_operator.py
@@ -601,7 +601,7 @@ def fhardsigmoid_grad(a, out_grad, alpha=0.2, beta=0.5):
 for dtype in [np.float16, np.float32, np.float64]:
 if dtype is np.float16:
 rtol = 1e-2
-atol = 1e-4
+atol = 1e-3
 else:
 rtol = 1e-3
 atol = 1e-5
@@ -611,9 +611,10 @@ def fhardsigmoid_grad(a, out_grad, alpha=0.2, beta=0.5):
 xa[xa == -2.5] = xa[xa == -2.5] - 1e-2
 ya = fhardsigmoid(xa)
 grad_xa = fhardsigmoid_grad(xa, np.ones(shape))
-check_numeric_gradient(y, [xa], numeric_eps=1e-3, rtol=rtol, atol=atol)
-check_symbolic_forward(y, [xa], [ya], rtol=rtol, atol=atol)
-check_symbolic_backward(y, [xa], [np.ones(shape)], [grad_xa], 
rtol=rtol, atol=atol)
+if dtype is not np.float16:
+check_numeric_gradient(y, [xa], numeric_eps=1e-3, rtol=rtol, 
atol=atol, dtype=dtype)
+check_symbolic_forward(y, [xa], [ya], rtol=rtol, atol=atol, 
dtype=dtype)
+check_symbolic_backward(y, [xa], [np.ones(shape)], [grad_xa], 
rtol=rtol, atol=atol, dtype=dtype)
 
 @with_seed()
 def test_softsign():


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on issue #10381: support profile can be saved to s3

[GitBox]

cjolivier01 commented on issue #10381: support profile can be saved to s3 
URL: https://github.com/apache/incubator-mxnet/pull/10381#issuecomment-384757585
 
 
   -0
   I don't care if you add this,but it's probably not such a great idea to try 
to save this stuff to s3. It's usually generated pretty quickly and it seems 
unlikely that s3 would be able to keep up.  But do it if you want.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10619: handle NDArray slice properly for mkldnn layout

[GitBox]

ashokei commented on issue #10619: handle NDArray slice properly for mkldnn 
layout
URL: https://github.com/apache/incubator-mxnet/pull/10619#issuecomment-384756746
 
 
   @zheng-da @piiswrong can you please merge this if ok, thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha opened a new pull request #10704: fix rnn layer repr

[GitBox]

szha opened a new pull request #10704: fix rnn layer repr
URL: https://github.com/apache/incubator-mxnet/pull/10704
 
 
   ## Description ##
   fix the problem of printing output dimension of 100.0 for gluon rnn layers
   
   ## Checklist ##
   ### Essentials ###
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

svn commit: r26560 - /dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz

[anirudh2290]

Author: anirudh2290
Date: Thu Apr 26 18:53:48 2018
New Revision: 26560

Log:
Add mxnet tar gz

Added:
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz  
 (with props)

Added: 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz
==
Binary file - no diff available.

Propchange: 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz
--
svn:mime-type = application/octet-stream

svn commit: r26559 - in /dev/incubator/mxnet/1.2.0.rc1: ./ apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.asc apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.md5 apache-mxnet-src-1.2.0.rc1-incubating.tar.g

[anirudh2290]

Author: anirudh2290
Date: Thu Apr 26 18:51:18 2018
New Revision: 26559

Log:
Add mxnet 1.2.0.rc1

Added:
dev/incubator/mxnet/1.2.0.rc1/

dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.asc

dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.md5

dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.sha512

Added: 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.asc
==
--- 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.asc 
(added)
+++ 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.asc 
Thu Apr 26 18:51:18 2018
@@ -0,0 +1,11 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQEcBAABAgAGBQJa4h5IAAoJEDjVoQyIJMHA8gsIALV9Yb5jdRDCXfVeptE0Dr0E
+znzMf1P/LeW51JsjSgoFAliaMhEpoyFPGyPq2pNKOseLijn3PgsYUHau91jzub/N
+XQvEjjLm9HRjdZnqU4R8w9TVPi6KEYj+hh+aOEWK+1hlkabhCYZCvoSZQJR7W6Fw
+K4SraR3xEOjzM4WP89W0E5ZqAOK+ycC5LrWlAHtNtgAw4TllysDtxU0W8Sroio2a
+vYbJjziFVpSW1chJ5m7V0hTYcLuNe3PRxeeKz5wu06QDqGWchYZzTQWaGPG3Tulj
+CoiEt90Yux3i26K0JHxPPEadhP3jEyN47k7mEKfACN9HaaUtgGolUbdSjlhzaCY=
+=L+fF
+-END PGP SIGNATURE-

Added: 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.md5
==
--- 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.md5 
(added)
+++ 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.md5 
Thu Apr 26 18:51:18 2018
@@ -0,0 +1 @@
+e8df0d7828c993128031f141f25c2580  apache-mxnet-src-1.2.0.rc1-incubating.tar.gz

Added: 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.sha512
==
--- 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.sha512
 (added)
+++ 
dev/incubator/mxnet/1.2.0.rc1/apache-mxnet-src-1.2.0.rc1-incubating.tar.gz.sha512
 Thu Apr 26 18:51:18 2018
@@ -0,0 +1 @@
+2f12b7d898420a861cd1d312fe130b283424d8a589208ae85c92206693970075b7c7913c2ac121b0dd7baaa55a7ce2f0e8fafe4a02774eac1ec199bd821881a7
  apache-mxnet-src-1.2.0.rc1-incubating.tar.gz

[GitHub] zhreshold commented on issue #8547: Revised validating process of a bounding box in ImageDetIter

[GitBox]

zhreshold commented on issue #8547: Revised validating process of a bounding 
box in ImageDetIter
URL: https://github.com/apache/incubator-mxnet/pull/8547#issuecomment-384750761
 
 
   @munkim Do you have any update on this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] bradsev closed pull request #10703: test edit

[GitBox]

bradsev closed pull request #10703: test edit
URL: https://github.com/apache/incubator-mxnet/pull/10703
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/README.md b/README.md
index ba37cd4bf83..acc2d23f124 100644
--- a/README.md
+++ b/README.md
@@ -68,7 +68,8 @@ Features
 * Support for Python, R, Scala, C++ and Julia
 * Cloud-friendly and directly compatible with S3, HDFS, and Azure
 
-Ask Questions
+Ask Questions - lots and lots
+of friendly queries
 -
 * Please use [discuss.mxnet.io](https://discuss.mxnet.io/) for asking 
questions.
 * Please use [mxnet/issues](https://github.com/dmlc/mxnet/issues) for 
reporting bugs.


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] bzcheeseman commented on issue #10340: Add PolyScheduler which mimics the caffe polynomial lr scheduler

[GitBox]

bzcheeseman commented on issue #10340: Add PolyScheduler which mimics the caffe 
polynomial lr scheduler
URL: https://github.com/apache/incubator-mxnet/pull/10340#issuecomment-384748303
 
 
   @piiswrong I can't figure out where my addition would have caused the error 
- unless I misunderstand it's confined to the cpp-package? It almost looks like 
docker is misconfigured...
   
   The error:
   ```
   [Python3: GPU] Cannot contact mxnet-linux-gpu21: 
java.lang.InterruptedException
   
   [Python3: GPU] build.py: 2018-03-31 12:27:55,935 Running of command in 
container failed: nvidia-docker run --rm -v 
/home/jenkins_slave/workspace/ut-python3-gpu:/work/mxnet -v 
/home/jenkins_slave/workspace/ut-python3-gpu/build:/work/build -u 1001:1001 
mxnet/build.ubuntu_gpu /work/runtime_functions.sh unittest_ubuntu_python3_gpu
   
   [Python3: GPU] build.py: 2018-03-31 12:27:55,935 You can try to get into the 
container by using the following command: nvidia-docker run --rm -v 
/home/jenkins_slave/workspace/ut-python3-gpu:/work/mxnet -v 
/home/jenkins_slave/workspace/ut-python3-gpu/build:/work/build -u 1001:1001 -ti 
--entrypoint bash mxnet/build.ubuntu_gpu /work/runtime_functions.sh 
unittest_ubuntu_python3_gpu
   
   [Python3: GPU] Traceback (most recent call last):
   
   [Python3: GPU]   File "ci/build.py", line 179, in 
   
   [Python3: GPU] sys.exit(main())
   
   [Python3: GPU]   File "ci/build.py", line 159, in main
   
   [Python3: GPU] container_run(platform, docker_binary, command)
   
   [Python3: GPU]   File "ci/build.py", line 110, in container_run
   
   [Python3: GPU] raise subprocess.CalledProcessError(ret, cmd)
   
   [Python3: GPU] subprocess.CalledProcessError: Command 'nvidia-docker run 
--rm -v /home/jenkins_slave/workspace/ut-python3-gpu:/work/mxnet -v 
/home/jenkins_slave/workspace/ut-python3-gpu/build:/work/build -u 1001:1001 
mxnet/build.ubuntu_gpu /work/runtime_functions.sh unittest_ubuntu_python3_gpu' 
returned non-zero exit status 137
   
   Click here to forcibly terminate running steps
   
   Click here to forcibly terminate running steps
   
   Click here to forcibly terminate running steps
   
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham closed pull request #10702: danga zone added

[GitBox]

aaronmarkham closed pull request #10702: danga zone added
URL: https://github.com/apache/incubator-mxnet/pull/10702
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/README.md b/README.md
index ba37cd4bf83..ef293cee815 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,9 @@
 Apache MXNet (incubating) for Deep Learning
 =
+My name is Aaron.
+
+Danga zone!
+
 
 [![Build 
Status](http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/job/master/badge/icon)](http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/job/master/)
 [![Documentation 
Status](http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet-build-site/badge/icon)](https://mxnet.incubator.apache.org/)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #10340: Add PolyScheduler which mimics the caffe polynomial lr scheduler

[GitBox]

piiswrong commented on issue #10340: Add PolyScheduler which mimics the caffe 
polynomial lr scheduler
URL: https://github.com/apache/incubator-mxnet/pull/10340#issuecomment-384746511
 
 
   @bzcheeseman ping could fix the tests?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] bradsev opened a new pull request #10703: test edit

[GitBox]

bradsev opened a new pull request #10703: test edit
URL: https://github.com/apache/incubator-mxnet/pull/10703
 
 
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #9841: Update versions of python dependencies

[GitBox]

piiswrong closed pull request #9841: Update versions of python dependencies
URL: https://github.com/apache/incubator-mxnet/pull/9841
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/setup.py b/python/setup.py
index cf94adf982d..9fb7f4d9850 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -28,7 +28,7 @@
 else:
 from setuptools import setup
 from setuptools.extension import Extension
-kwargs = {'install_requires': ['numpy<=1.13.3,>=1.8.2', 
'requests==2.18.4', 'graphviz==0.8.1'], 'zip_safe': False}
+kwargs = {'install_requires': ['numpy<1.15.0,>=1.8.2', 
'requests<2.19.0,>=2.18.4', 'graphviz<0.9.0,>=0.8.1'], 'zip_safe': False}
 from setuptools import find_packages
 
 with_cython = False
diff --git a/tests/ci_build/install/ubuntu_install_python.sh 
b/tests/ci_build/install/ubuntu_install_python.sh
index 0ba98004a3e..12955714ece 100755
--- a/tests/ci_build/install/ubuntu_install_python.sh
+++ b/tests/ci_build/install/ubuntu_install_python.sh
@@ -25,5 +25,5 @@ apt-get update && apt-get install -y python-dev python3-dev
 # the version of the pip shipped with ubuntu may be too lower, install a 
recent version here
 cd /tmp && wget https://bootstrap.pypa.io/get-pip.py && python3 get-pip.py && 
python2 get-pip.py
 
-pip2 install nose pylint numpy nose-timer requests h5py scipy
-pip3 install nose pylint numpy nose-timer requests h5py scipy
+pip2 install nose pylint 'numpy<1.15.0,>=1.8.2' nose-timer 
'requests<2.19.0,>=2.18.4' h5py scipy
+pip3 install nose pylint 'numpy<1.15.0,>=1.8.2' nose-timer 
'requests<2.19.0,>=2.18.4' h5py scipy


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #9841: Update versions of python dependencies

[GitBox]

piiswrong commented on issue #9841: Update versions of python dependencies
URL: https://github.com/apache/incubator-mxnet/pull/9841#issuecomment-384745957
 
 
   looks like another PR fixed this. Closing


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #9558: Signum with grad compression

[GitBox]

piiswrong closed pull request #9558: Signum with grad compression
URL: https://github.com/apache/incubator-mxnet/pull/9558
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/kvstore/gradient_compression-inl.h 
b/src/kvstore/gradient_compression-inl.h
index 9b69bd11472..3d9c200ffca 100644
--- a/src/kvstore/gradient_compression-inl.h
+++ b/src/kvstore/gradient_compression-inl.h
@@ -36,6 +36,10 @@ void Quantize2BitImpl(mshadow::Stream *s, 
const std::vector *s, const 
std::vector ,
 const float threshold);
+void QuantizeSignumImpl(mshadow::Stream *s, const 
std::vector ,
+  const float beta);
+void DequantizeSignumImpl(mshadow::Stream *s,
+  const std::vector );
 
 struct quantize_2bit {
   MSHADOW_XINLINE static void Map(int out_block_id,
@@ -94,6 +98,83 @@ void Quantize2BitKernelLaunch(mshadow::Stream *s, const 
std::vector> 2);
+// start and end are indices in original grad array
+
+// by 4 into 32 = into 8
+const int start = out_byte_id << 3;
+const int end = (start + 8 <= original_size) ? start + 8 : original_size;
+// cast as char* to manipulate bits of float addresses
+unsigned char *block_ptr = reinterpret_cast < unsigned char * > 
(compr_block) + (out_byte_id & 3);
+*block_ptr = 0;
+float* res = residual + start;
+float* g = grad + start;
+uint8_t mask = 1U << 7;
+for (int i = start; i < end; i++) {
+  // adds offset to reach appropriate byte
+  // adds gradient to existing residual to get updated grad
+  *res = (*res * beta) + (oneminusbeta * (*g++));
+  if (*res++ >= 0) {
+*block_ptr |= mask;
+  }
+  mask >>= 1;
+}
+  }
+};
+
+template
+void QuantizeSignumKernelLaunch(mshadow::Stream *s, const 
std::vector ,
+  const float beta) {
+  mxnet::op::mxnet_op::Kernel
+::Launch(s,
+inputs[2].Size() * 4, // compressed array size
+inputs[0].Size(), // original size
+inputs[2].dptr(),  // compressed array
+inputs[0].dptr(),  // original array
+inputs[1].dptr(),  // residual array
+beta, 1-beta);   // positive threshold
+}
+
+struct dequantize_signum {
+  MSHADOW_XINLINE static void Map(int i,
+  float *out,
+  float *in) {
+// get position of dequantized value to fill
+float *outval = out + i;
+// gets byte which holds quantized value for this position
+unsigned char *ch_ptr = reinterpret_cast(in + (i >> 5));
+ch_ptr += ((i & 31) >> 3);
+
+if (*ch_ptr & (1U << (7 - ( i & 7  {
+  *outval = 1;
+} else {
+  *outval = -1;
+}
+  }
+};
+
+template
+void DequantizeSignumKernelLaunch(mshadow::Stream *s,
+  const std::vector ) {
+  mxnet::op::mxnet_op::Kernel
+  ::Launch(s,
+  inputs[1].Size(), // original size
+  inputs[1].dptr(),  // out array
+  inputs[0].dptr());  // compressed array
+}
+
+
 struct dequantize_2bit {
   MSHADOW_XINLINE static void Map(int i,
   float *out,
@@ -149,6 +230,18 @@ inline void 
Dequantize2BitImpl(mshadow::Stream *s,
const float threshold) {
   Dequantize2BitKernelLaunch(s, inputs, threshold);
 }
+
+inline void QuantizeSignumImpl(mshadow::Stream *s,
+ const std::vector ,
+ const float beta) {
+  QuantizeSignumKernelLaunch(s, inputs, beta);
+}
+
+inline void DequantizeSignumImpl(mshadow::Stream *s,
+   const std::vector ) {
+  DequantizeSignumKernelLaunch(s, inputs);
+}
+
 }  // namespace kvstore
 }  // namespace mxnet
 
diff --git a/src/kvstore/gradient_compression.cc 
b/src/kvstore/gradient_compression.cc
index b8c626cd53a..9feeab13d3a 100644
--- a/src/kvstore/gradient_compression.cc
+++ b/src/kvstore/gradient_compression.cc
@@ -61,6 +61,8 @@ void GradientCompression::SetParams(const 
std::vector

[GitHub] aaronmarkham commented on issue #10702: danga zone added

[GitBox]

aaronmarkham commented on issue #10702: danga zone added
URL: https://github.com/apache/incubator-mxnet/pull/10702#issuecomment-384745632
 
 
   @awschris
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #9947: [WIP] Performance optimization for dot(csr, rsp) on cpu

[GitBox]

piiswrong commented on issue #9947: [WIP] Performance optimization for dot(csr, 
rsp) on cpu
URL: https://github.com/apache/incubator-mxnet/pull/9947#issuecomment-384745271
 
 
   @ZiyueHuang @eric-haibin-lin 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham opened a new pull request #10702: danga zone added

[GitBox]

aaronmarkham opened a new pull request #10702: danga zone added
URL: https://github.com/apache/incubator-mxnet/pull/10702
 
 
   ## Description ##
   danja zone
   
   ## Checklist ##
   ### Essentials ###
   
   - [x] Changes are complete (i.e. I finished coding on this PR)
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #10297: [MXNET-244][WIP Don't merge] Fixes for cross compilation in ARM

[GitBox]

piiswrong commented on issue #10297: [MXNET-244][WIP Don't merge] Fixes for 
cross compilation in ARM
URL: https://github.com/apache/incubator-mxnet/pull/10297#issuecomment-384745097
 
 
   @larroy ping


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #9933: [MXNET-23] Adding support to profile kvstore server during distributed training

[GitBox]

piiswrong commented on issue #9933: [MXNET-23] Adding support to profile 
kvstore server during distributed training
URL: https://github.com/apache/incubator-mxnet/pull/9933#issuecomment-384744930
 
 
   @rahul003 ping


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #9906: Add CPU optimized docker which will be compiled with MKL-DNN

[GitBox]

piiswrong commented on issue #9906: Add CPU optimized docker which will be 
compiled with MKL-DNN
URL: https://github.com/apache/incubator-mxnet/pull/9906#issuecomment-384744629
 
 
   closing due to inactivity. looks like @marcoabreu thinks we don't need this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #9906: Add CPU optimized docker which will be compiled with MKL-DNN

[GitBox]

piiswrong closed pull request #9906: Add CPU optimized docker which will be 
compiled with MKL-DNN
URL: https://github.com/apache/incubator-mxnet/pull/9906
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docker/Dockerfiles/Dockerfile.in.lib.mkl 
b/docker/Dockerfiles/Dockerfile.in.lib.mkl
new file mode 100755
index 000..885247be2e3
--- /dev/null
+++ b/docker/Dockerfiles/Dockerfile.in.lib.mkl
@@ -0,0 +1,10 @@
+# -*- mode: dockerfile -*-
+# dockerfile to build libmxnet.so on CPU with MKL
+FROM ubuntu:14.04
+
+COPY install/cpp.sh install/
+RUN install/cpp.sh
+
+RUN git clone --recursive https://github.com/dmlc/mxnet && cd mxnet && \
+make -j$(nproc) USE_MKLDNN=1 && \
+rm -r build


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #10242: [MXNET-137]fix parameters name inconsistent for Proposal OP and Multi Proposal OP

[GitBox]

piiswrong commented on issue #10242: [MXNET-137]fix parameters name 
inconsistent for Proposal OP and Multi Proposal OP
URL: https://github.com/apache/incubator-mxnet/pull/10242#issuecomment-384743810
 
 
   @wkcn ping


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services