[GitHub] lanking520 commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

lanking520 commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-395654809
 
 
   @lupesko 
   Content from the image:
   Maximum CPU Consumption: 158.13   Average CPU Consumption: 109.02
   Maximum CPU Usage: 100 Average CPU Usage: 89.76
   Maximum temperature in core: 62 Maximum Core Frequency 1860
   使用mxnet-cu91版本的时候内存占用会高很多到5899M，但是耗时更短只要381秒左右。。。而且使用率也高很多，接近99%
   When I use mxnet-cu91 verion, the occupation for the memory can be up to 
5899M, however time is great less, around 381 seconds... And the usability rate 
is very high, close to 99%
   跑的是vgg-gluon的教程。。。
   I run the tutorial in vgg-gluon
   
   Follow up conversation:
   
   cu91是1.2beta，cu92是6月2号的版本 
   还有cu92说要linux的nVidia驱动的396.26版本才支持，实际上现在发布的只有396.24？？396.26是不是打错了？
   cu91 is 1.2beta, cu92 is a version released in 6.2,
   and cu92 supposed to be supported by NVIDIA Driver 396.26, however the 
highest version we have is only 396.24? Is 396.26 a typo?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

lanking520 commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-395654809
 
 
   @lupesko 
   Content from the image:
   Maximum CPU Consumption: 158.13   Average CPU Consumption: 109.02
   Maximum CPU Usage: 100 Average CPU Usage: 89.76
   Maximum temperature in core: 62 Maximum Core Frequency 1860
   使用mxnet-cu91版本的时候内存占用会高很多到5899M，但是耗时更短只要381秒左右。。。而且使用率也高很多，接近99%
   When I use mxnet-cu91 verion, the occupation for the memory can be up to 
5899M, however time is great less, around 381 seconds... And the usability rate 
is very high, close to 99%
   跑的是vgg-gluon的教程。。。
   I run the tutorial in vgg-gluon
   
   Follow up conversation:
   
   cu91是1.2beta，cu92是6月2号的版本 
   还有cu92说要linux的nVidia驱动的396.26版本才支持，实际上现在发布的只有396.24？？396.26是不是打错了？
   cu91 is 1.2beta, cu92 is a version in 6.2,
   and cu92 supposed to be supported by NVIDIA Driver 396.26, however the 
highest version we have is only 396.24? Is 396.26 a typo?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

lanking520 commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-395654809
 
 
   @lupesko 
   Content from the image:
   Maximum CPU Consumption: 158.13   Average CPU Consumption: 109.02
   Maximum CPU Usage: 100 Average CPU Usage: 89.76
   Maximum temperature in core: 62 Maximum Core Frequency 1860
   使用mxnet-cu91版本的时候内存占用会高很多到5899M，但是耗时更短只要381秒左右。。。而且使用率也高很多，接近99%
   When I use mxnet-cu91 verion, the occupation for the memory can be up to 
5899M, however time is great less, around 381 seconds... And the usability rate 
is very high, close to 99%
   跑的是vgg-gluon的教程。。。
   I run the tutorial in vgg-gluon
   
   Follow up conversation:
   
   cu91是1.2beta，cu92是6月2号的版本 
   还有cu92说要linux的nVidia驱动的396.26版本才支持，实际上现在发布的只有396.24？？396.26是不是打错了？
   cu91 is 1.2beta, cu92 is a version in 6.2,
   and cu92 supposed to be supported by NVIDIA Driver 396.26, however the 
highest version we have is only 396.24? Is 396.26 a mistake?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lupesko commented on issue #11131: mxnet-cu92 low efficiency

[GitBox]

lupesko commented on issue #11131: mxnet-cu92 low efficiency
URL: 
https://github.com/apache/incubator-mxnet/issues/11131#issuecomment-395653697
 
 
   @lanking520 can you please add a translation of the issue? My putonghua is a 
bit rusty 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 commented on issue #11163: Deadlock in LibMxNet.dll during unload due to synchronization call in destructor

[GitBox]

anirudh2290 commented on issue #11163: Deadlock in LibMxNet.dll during unload 
due to synchronization call in destructor 
URL: 
https://github.com/apache/incubator-mxnet/issues/11163#issuecomment-395650222
 
 
   Related to #8921 and #9271


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] HuichuanLiu commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py

[GitBox]

HuichuanLiu commented on issue #11149: Unreasonable performance of resnext 
models provided in model_zoo, evaluated by score.py
URL: 
https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-395647325
 
 
   @lanking520 Right, I've tried it with mxnet-cu80 1.2.0 from pip:
   ```
   [12:41:56] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: 
/data3/liuhuichuan/Data/imagenet/imagenet1k-val.rec, use 4 threads for 
decoding..
   [12:42:01] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by 
previous version v0.9.4. Attempting to upgrade...
   [12:42:01] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
   [12:42:06] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running 
performance tests to find the best convolution algorithm, this can take a 
while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   INFO:root:Finished with 160.141619 images per second
   INFO:root:('accuracy', 0.48345588235294118)
   INFO:root:('top_k_accuracy_5', 0.6939338235294118)
   ```
   
   2. Will build from source later, but it takes some time
   
   3. Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-62-generic x86_64)
   
   Thanks for your help : )


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin opened a new pull request #11197: [WIP] Gluon sparse block and sparse embedding

[GitBox]

eric-haibin-lin opened a new pull request #11197: [WIP] Gluon sparse block and 
sparse embedding
URL: https://github.com/apache/incubator-mxnet/pull/11197
 
 
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] HuichuanLiu commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py

[GitBox]

HuichuanLiu commented on issue #11149: Unreasonable performance of resnext 
models provided in model_zoo, evaluated by score.py
URL: 
https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-395643218
 
 
   @lanking520 Right, I've tried mxnet-cu80 1.2.0(but from pip) and the result:
   ```
   [11:52:32] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: 
/data3/liuhuichuan/Data/imagenet/imagenet1k-val.rec, use 4 threads for 
decoding..
   [11:52:38] src/nnvm/legacy_json_util.cc:190: Loading symbol saved by 
previous version v0.8.0. Attempting to upgrade...
   [11:52:38] src/nnvm/legacy_json_util.cc:198: Symbol successfully upgraded!
   [11:52:41] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running 
performance tests to find the best convolution algorithm, this can take a 
while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   INFO:root:Finished with 270.369418 images per second
   INFO:root:('accuracy', 7.9923273657289009e-05)
   INFO:root:('top_k_accuracy_5', 0.0022778132992327367)
   ```
   
   2. Will try to build from source later but it could take some time, like 2 
days.
   
   3. OS info : Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-62-generic x86_64)
   
   Thanks for your help：-）


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] HuichuanLiu commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py

[GitBox]

HuichuanLiu commented on issue #11149: Unreasonable performance of resnext 
models provided in model_zoo, evaluated by score.py
URL: 
https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-395643218
 
 
   @lanking520 Right, I've tried mxnet-cu80 1.2.0(but from pip) and the result:
   ```
   [11:52:32] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: 
/data3/liuhuichuan/Data/imagenet/imagenet1k-val.rec, use 4 threads for 
decoding..
   [11:52:38] src/nnvm/legacy_json_util.cc:190: Loading symbol saved by 
previous version v0.8.0. Attempting to upgrade...
   [11:52:38] src/nnvm/legacy_json_util.cc:198: Symbol successfully upgraded!
   [11:52:41] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running 
performance tests to find the best convolution algorithm, this can take a 
while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   INFO:root:Finished with 270.369418 images per second
   INFO:root:('accuracy', 7.9923273657289009e-05)
   INFO:root:('top_k_accuracy_5', 0.0022778132992327367)
   ```
   
   2. Will try to build from source later but it could take some time, like 2 
days.
   
   3. OS info : Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-62-generic x86_64)
   
   Thanks for your help：-）


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zheng-da commented on a change in pull request #11194: Enable CUDNN for conv1D

[GitBox]

zheng-da commented on a change in pull request #11194: Enable CUDNN for conv1D
URL: https://github.com/apache/incubator-mxnet/pull/11194#discussion_r193948100
 
 

 ##
 File path: src/operator/nn/convolution.cc
 ##
 @@ -384,12 +384,9 @@ struct ConvolutionGrad {
   const char *op_name;
   std::vector operator()(const nnvm::NodePtr& n,
   const std::vector& 
ograds) const {
-const ConvolutionParam& param = 
nnvm::get(n->attrs.parsed);
 std::vector heads(ograds.begin(), ograds.end());
 heads.push_back(n->inputs[conv::kData]);
 heads.push_back(n->inputs[conv::kWeight]);
-if (!param.no_bias)
-  heads.push_back(n->inputs[conv::kBias]);
 
 Review comment:
   This is needed for MKLDNN.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py

[GitBox]

lanking520 commented on issue #11149: Unreasonable performance of resnext 
models provided in model_zoo, evaluated by score.py
URL: 
https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-395639961
 
 
   @HuichuanLiu Sorry for that. I do see you are using anaconda with MXNet 1.1.
   Have you tried build from source? 
   Which Ubuntu machine are you using? 
   I will try to reproduce the problem you have using a Linux Instance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py

[GitBox]

lanking520 commented on issue #11149: Unreasonable performance of resnext 
models provided in model_zoo, evaluated by score.py
URL: 
https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-395639961
 
 
   @HuichuanLiu Sorry for that. I do see you are using anaconda with MXNet 1.1.
   Have you tried build from source using git? 
   Which Ubuntu machine are you using? 
   I will try to reproduce the problem you have using a Linux Instance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Alexander-flyer closed issue #10596: Error :"cannot find -lcudnn" when compile mxnet

[GitBox]

Alexander-flyer closed issue #10596: Error :"cannot find -lcudnn"  when compile 
mxnet
URL: https://github.com/apache/incubator-mxnet/issues/10596
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] safrooze commented on issue #11194: Enable CUDNN for conv1D

[GitBox]

safrooze commented on issue #11194: Enable CUDNN for conv1D
URL: https://github.com/apache/incubator-mxnet/pull/11194#issuecomment-395633916
 
 
   @eric-haibin-lin CI failed. Can you take a look? This is the error:
   
   simple_bind failed: Error: Error in operator convolution0_backward: 
[02:38:11] src/operator/nn/convolution.cc:320: Check failed: in_attrs->size() 
== in_expected (3 vs. 4)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jinhuang415 commented on issue #10433: [MXNET-290] MKLDNN support for model quantization

[GitBox]

jinhuang415 commented on issue #10433: [MXNET-290] MKLDNN support for model 
quantization
URL: https://github.com/apache/incubator-mxnet/pull/10433#issuecomment-395630010
 
 
   @marcoabreu I used my proposed approach 1 to test all cases and skip and 
print the cases that is expected to skip, please help to take a look. Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kpmurali commented on issue #11190: [MXNET-529] Add version select in the API pages

[GitBox]

kpmurali commented on issue #11190: [MXNET-529] Add version select in the API 
pages
URL: https://github.com/apache/incubator-mxnet/pull/11190#issuecomment-395628023
 
 
   Closed in favour of PR - https://github.com/apache/incubator-mxnet/pull/11196


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kpmurali closed pull request #11190: [MXNET-529] Add version select in the API pages

[GitBox]

kpmurali closed pull request #11190: [MXNET-529] Add version select in the API 
pages
URL: https://github.com/apache/incubator-mxnet/pull/11190
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/_static/js/versions_drop_down.js 
b/docs/_static/js/versions_drop_down.js
new file mode 100644
index 000..4ab2abf2afa
--- /dev/null
+++ b/docs/_static/js/versions_drop_down.js
@@ -0,0 +1,15 @@
+$(document).ready(function () {
+  function loadVersionURL() {
+  let el = $(this);
+  console.log("loadVersionURL");
+  let versionString = '';
+  if ($(this).text().includes("master")) {
+  versionString = $(this).text();
+  } else {
+  //Remove the character v at the beginning
+  versionString = $(this).text().substr(1);
+  }
+  window.location.pathname = '/versions/' + versionString + 
window.location.pathname ;
+  }
+  $('.opt-group').on('click', '.versions', loadVersionURL);
+});
diff --git a/docs/_static/mxnet-theme/layout.html 
b/docs/_static/mxnet-theme/layout.html
index 3028342abd5..d72582cc1c7 100644
--- a/docs/_static/mxnet-theme/layout.html
+++ b/docs/_static/mxnet-theme/layout.html
@@ -102,11 +102,17 @@
must come *after* these tags. #}
 {{ metatags }}
 {%- block htmltitle %}
-{%- if pagename != 'index' and 'no title' not in title%}
+
+https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/og-logo.png;
 />
+https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/og-logo.png;
 />
+{%- if pagename != 'index' and 'no title' not in title %}
+
 {{ title|striptags|e }}{{ titlesuffix }}
 {%- elif pagename == 'index' %}
+
 MXNet: A Scalable Deep Learning Framework
 {%- else %}
+
 {{ pagename.split('/')[0]|capitalize }}{{ titlesuffix }}
 {%- endif %}
 {%- endblock %}
diff --git a/docs/api/c++/index.md b/docs/api/c++/index.md
index 9a68cdafda6..ff7877ba89f 100644
--- a/docs/api/c++/index.md
+++ b/docs/api/c++/index.md
@@ -1,4 +1,18 @@
 # MXNet - C++ API
+
+  
+v1.2.0
+
+
+  v1.2.0
+  v1.1.0
+  v1.0.0
+  v0.12.1
+  v0.11.0
+  master
+
+  
+
 
 For namespaces, classes, and code files for the MXNet C++ package, see the  
following:
 
diff --git a/docs/api/julia/index.md b/docs/api/julia/index.md
index f4dc0e5b749..9a9be91f769 100644
--- a/docs/api/julia/index.md
+++ b/docs/api/julia/index.md
@@ -1,4 +1,18 @@
 # MXNet - Julia API
+
+  
+v1.2.0
+
+
+  v1.2.0
+  v1.1.0
+  v1.0.0
+  v0.12.1
+  v0.11.0
+  master
+
+  
+
 
 See the [MXNet Julia Reference 
Manual](https://media.readthedocs.org/pdf/mxnet-test/latest/mxnet-test.pdf).
 
diff --git a/docs/api/perl/index.md b/docs/api/perl/index.md
index 6408b525f00..27e39db1e16 100644
--- a/docs/api/perl/index.md
+++ b/docs/api/perl/index.md
@@ -1,4 +1,18 @@
 # MXNet - Perl API
+
+  
+v1.2.0
+
+
+  v1.2.0
+  v1.1.0
+  v1.0.0
+  v0.12.1
+  v0.11.0
+  master
+
+  
+
 
 MXNet supports the Perl programming language. The MXNet Perl package brings 
flexible and efficient GPU
 computing and state-of-art deep learning to Perl. It enables you to write 
seamless tensor/matrix computation with multiple GPUs in Perl.
diff --git a/docs/api/python/index.md b/docs/api/python/index.md
index 420f4c9b72f..7e8891f9988 100644
--- a/docs/api/python/index.md
+++ b/docs/api/python/index.md
@@ -1,5 +1,20 @@
 # MXNet - Python API
 
+
+
+  
+v1.2.0
+
+
+  v1.2.0
+  v1.1.0
+  v1.0.0
+  v0.12.1
+  v0.11.0
+  master
+
+  
+
 MXNet provides a comprehensive and flexible Python API to serve a broad 
community of developers with different levels of experience and wide ranging 
requirements. In this section, we provide an in-depth discussion of the 
functionality provided by various MXNet Python packages.
 
 MXNet's Python API has two primary high-level packages*: the Gluon API and 
Module API. We recommend that new users start with the Gluon API as it's more 
flexible and easier to debug. Underlying these high-level packages are the core 
packages of NDArray and Symbol.
diff --git a/docs/api/r/index.md b/docs/api/r/index.md
index 822c59dbe3c..0bedc5dab82 100644
--- a/docs/api/r/index.md
+++ b/docs/api/r/index.md
@@ -1,4 +1,18 @@
 # MXNet - R API
+
+  
+v1.2.0
+
+
+  v1.2.0
+  v1.1.0
+  v1.0.0
+  v0.12.1
+  v0.11.0
+  master
+
+  
+
 
 See the [MXNet R Reference 
Manual](https://s3.amazonaws.com/mxnet-prod/docs/R/mxnet-r-reference-manual.pdf).
 
diff --git a/docs/api/scala/index.md b/docs/api/scala/index.md
index e96892b5800..3c8e640a573 100644
--- a/docs/api/scala/index.md

[GitHub] kpmurali opened a new pull request #11196: [MXNET-529] Adding the version selector to the API pages

[GitBox]

kpmurali opened a new pull request #11196: [MXNET-529] Adding the version 
selector to the API pages
URL: https://github.com/apache/incubator-mxnet/pull/11196
 
 
   ## Description ##
   Adding the versions drop-down to all the API pages
   
   ## Checklist ##
   ### Changes ###
   - [ x ] Add the version selector to the API pages
   - [ x ] Add javascript logic for changing the API pages according to the 
version
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] HuichuanLiu commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py

[GitBox]

HuichuanLiu commented on issue #11149: Unreasonable performance of resnext 
models provided in model_zoo, evaluated by score.py
URL: 
https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-395626744
 
 
   @lanking520 Well, it is not a HOW TO question, but more like a potential 
Problem in the checkpoint or scoring process. There're very few extra codes and 
I'm like just cloned the project and run the given scripts. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)

[GitBox]

eric-haibin-lin commented on issue #11142: [MXNET-408] inplace ReLU activation 
(#10847)
URL: https://github.com/apache/incubator-mxnet/pull/11142#issuecomment-395623882
 
 
   @nswamy do i only need to handle non-scale version bumps now? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new e9dcf54  Bump the publish timestamp.
e9dcf54 is described below

commit e9dcf549931e2e974ccf5d48ad33767fc06b2a09
Author: mxnet-ci 
AuthorDate: Fri Jun 8 02:10:21 2018 +

Bump the publish timestamp.
---
 date.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/date.txt b/date.txt
new file mode 100644
index 000..4ec2fed
--- /dev/null
+++ b/date.txt
@@ -0,0 +1 @@
+Fri Jun  8 02:10:21 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] ThomasDelteil commented on issue #10827: [MXNET-405][WIP] Add 2 new pipelines to the Official CI and run nightly tests.

[GitBox]

ThomasDelteil commented on issue #10827: [MXNET-405][WIP] Add 2 new pipelines 
to the Official CI and run nightly tests. 
URL: https://github.com/apache/incubator-mxnet/pull/10827#issuecomment-395622983
 
 
   Can we have the tutorials test added to the nightly build ? (removed here) 
https://github.com/apache/incubator-mxnet/pull/11170/files


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] Feywell commented on issue #11150: Hope for adding sample about mx.rnn.ConvGRUCell()

[GitBox]

Feywell commented on issue #11150: Hope for adding sample about 
mx.rnn.ConvGRUCell()
URL: 
https://github.com/apache/incubator-mxnet/issues/11150#issuecomment-395620473
 
 
   @lanking520 Thanks for your reply! I know that is not a bug in MXNet. But I 
can not find enough documents of ConvRNN about MXNet implements. 
   Thanks for your help, again  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] glingyan opened a new issue #11195: onnx do not support conv weight from symbol

[GitBox]

glingyan opened a new issue #11195: onnx do not support conv weight from symbol
URL: https://github.com/apache/incubator-mxnet/issues/11195
 
 
   if weight is from another node
   will raise exception
   def _fix_channels(op_name, attrs, inputs, proto_obj):
   """A workaround for getting 'channels' or 'units' since onnx don't 
provide
   these attributes. We check the shape of weights provided to get the 
number.
   """
   weight_name = inputs[1].name
   if not weight_name in proto_obj._params:
   **raise ValueError("Unable to get channels/units attr from onnx 
graph.")**
   
   
![image](https://user-images.githubusercontent.com/22489989/41134562-98fb2c40-6aff-11e8-9fe0-03b73e97f384.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #11194: [WIP] Enable CUDNN for conv1D

[GitBox]

eric-haibin-lin commented on issue #11194: [WIP] Enable CUDNN for conv1D
URL: https://github.com/apache/incubator-mxnet/pull/11194#issuecomment-395617406
 
 
   @zheng-da pls review


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing 
maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126#discussion_r193930733
 
 

 ##
 File path: 
scala-package/macros/src/main/scala/org/apache/mxnet/SymbolMacro.scala
 ##
 @@ -118,17 +119,28 @@ private[mxnet] object SymbolImplMacros {
 else {
   argDef += s"${currArgName} : ${symbolarg.argType}"
 }
-var base = "map(\"" + symbolarg.argName + "\") = " + currArgName
-if (symbolarg.isOptional) {
-  base = "if (!" + currArgName + ".isEmpty)" + base + ".get"
+// Symbol arg implementation
+val returnType = "org.apache.mxnet.Symbol"
+var base = ""
 
 Review comment:
   better code convention in Scala is 
   ```scala
   val base = 
  if ...
  else ...
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing 
maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126#discussion_r193928211
 
 

 ##
 File path: 
scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
 ##
 @@ -97,9 +97,11 @@ private[mxnet] object APIDocGenerator{
   argDef += "name : String = null"
   argDef += "attr : Map[String, String] = null"
 } else {
+  argDef += "out : Option[NDArray] = None"
   returnType = "org.apache.mxnet.NDArrayFuncReturn"
 }
-s"def ${func.name} (${argDef.mkString(", ")}) : ${returnType}"
+val experimentalTag = "@Experimental"
 
 Review comment:
   annotation is enough. Ensuring users realize it during coding is enough.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing 
maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126#discussion_r193929378
 
 

 ##
 File path: 
scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
 ##
 @@ -97,9 +97,11 @@ private[mxnet] object APIDocGenerator{
   argDef += "name : String = null"
   argDef += "attr : Map[String, String] = null"
 } else {
+  argDef += "out : Option[NDArray] = None"
   returnType = "org.apache.mxnet.NDArrayFuncReturn"
 }
-s"def ${func.name} (${argDef.mkString(", ")}) : ${returnType}"
+val experimentalTag = "@Experimental"
 
 Review comment:
   warning log is for runtime, e.g., one try to calculate something but got 
NaN, which cannot be detected during coding / compile time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing maintenance on NDArray

[GitBox]

yzhliu commented on a change in pull request #11126: [MXNET-386] ongoing 
maintenance on NDArray
URL: https://github.com/apache/incubator-mxnet/pull/11126#discussion_r193929148
 
 

 ##
 File path: 
scala-package/macros/src/main/scala/org/apache/mxnet/SymbolMacro.scala
 ##
 @@ -104,6 +104,7 @@ private[mxnet] object SymbolImplMacros {
   // Construct Implementation field
   var impl = ListBuffer[String]()
   impl += "val map = scala.collection.mutable.Map[String, Any]()"
+  impl += s"var args = Seq[org.apache.mxnet.Symbol]()"
 
 Review comment:
   `s` is unnecessary.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11186: Devel arm

[GitBox]

marcoabreu commented on a change in pull request #11186: Devel arm
URL: https://github.com/apache/incubator-mxnet/pull/11186#discussion_r193930619
 
 

 ##
 File path: Jenkinsfile
 ##
 @@ -445,6 +445,7 @@ try {
   timeout(time: max_time, unit: 'MINUTES') {
 init_git()
 docker_run('armv7', 'build_armv7', false)
+archiveArtifacts artifacts: 'build/*.whl,build/libmxnet.*'
 
 Review comment:
   I'd like to prevent the usage of archive artifacts as it is going to 
increase the permanently stored size of every run by a LOT - and it won't be 
possible to distinguish the different wheels from each other.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: [MXNET-525] Add retry logic to download functions to fix flaky tests (#11181)

[marcoabreu]

This is an automated email from the ASF dual-hosted git repository.

marcoabreu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new b434b8e  [MXNET-525] Add retry logic to download functions to fix 
flaky tests (#11181)
b434b8e is described below

commit b434b8ec18f774c99b0830bd3ca66859212b4911
Author: Thomas Delteil 
AuthorDate: Thu Jun 7 18:12:58 2018 -0700

[MXNET-525] Add retry logic to download functions to fix flaky tests 
(#11181)

* Adding retry logic to the download function

* Adding retry logic on download

* Fixing retry wording

* addressing feedback

* forgot parenthesis
---
 python/mxnet/gluon/utils.py   | 45 ---
 python/mxnet/test_utils.py| 31 -
 tests/python/unittest/test_gluon_utils.py | 34 +++
 tests/python/unittest/test_test_utils.py  | 34 +++
 4 files changed, 121 insertions(+), 23 deletions(-)

diff --git a/python/mxnet/gluon/utils.py b/python/mxnet/gluon/utils.py
index 818aa3d..06b91fa 100644
--- a/python/mxnet/gluon/utils.py
+++ b/python/mxnet/gluon/utils.py
@@ -171,7 +171,7 @@ def check_sha1(filename, sha1_hash):
 return sha1.hexdigest() == sha1_hash
 
 
-def download(url, path=None, overwrite=False, sha1_hash=None):
+def download(url, path=None, overwrite=False, sha1_hash=None, retries=5):
 """Download an given URL
 
 Parameters
@@ -186,6 +186,8 @@ def download(url, path=None, overwrite=False, 
sha1_hash=None):
 sha1_hash : str, optional
 Expected sha1 hash in hexadecimal digits. Will ignore existing file 
when hash is specified
 but doesn't match.
+retries : integer, default 5
+The number of times to attempt the download in case of failure or non 
200 return codes
 
 Returns
 ---
@@ -200,26 +202,37 @@ def download(url, path=None, overwrite=False, 
sha1_hash=None):
 fname = os.path.join(path, url.split('/')[-1])
 else:
 fname = path
+assert retries >= 0, "Number of retries should be at least 0"
 
 if overwrite or not os.path.exists(fname) or (sha1_hash and not 
check_sha1(fname, sha1_hash)):
 dirname = os.path.dirname(os.path.abspath(os.path.expanduser(fname)))
 if not os.path.exists(dirname):
 os.makedirs(dirname)
-
-print('Downloading %s from %s...'%(fname, url))
-r = requests.get(url, stream=True)
-if r.status_code != 200:
-raise RuntimeError("Failed downloading url %s"%url)
-with open(fname, 'wb') as f:
-for chunk in r.iter_content(chunk_size=1024):
-if chunk: # filter out keep-alive new chunks
-f.write(chunk)
-
-if sha1_hash and not check_sha1(fname, sha1_hash):
-raise UserWarning('File {} is downloaded but the content hash does 
not match. ' \
-  'The repo may be outdated or download may be 
incomplete. ' \
-  'If the "repo_url" is overridden, consider 
switching to ' \
-  'the default repo.'.format(fname))
+while retries+1 > 0:
+# Disable pyling too broad Exception
+# pylint: disable=W0703
+try:
+print('Downloading %s from %s...'%(fname, url))
+r = requests.get(url, stream=True)
+if r.status_code != 200:
+raise RuntimeError("Failed downloading url %s"%url)
+with open(fname, 'wb') as f:
+for chunk in r.iter_content(chunk_size=1024):
+if chunk: # filter out keep-alive new chunks
+f.write(chunk)
+if sha1_hash and not check_sha1(fname, sha1_hash):
+raise UserWarning('File {} is downloaded but the content 
hash does not match.'\
+  ' The repo may be outdated or download 
may be incomplete. '\
+  'If the "repo_url" is overridden, 
consider switching to '\
+  'the default repo.'.format(fname))
+break
+except Exception as e:
+retries -= 1
+if retries <= 0:
+raise e
+else:
+print("download failed, retrying, {} attempt{} left"
+  .format(retries, 's' if retries > 1 else ''))
 
 return fname
 
diff --git a/python/mxnet/test_utils.py b/python/mxnet/test_utils.py
index bcdcc9c..686802d 100644
--- a/python/mxnet/test_utils.py
+++ b/python/mxnet/test_utils.py
@@ -1367,7 +1367,7 @@ def list_gpus():
 pass
 return range(len([i for i in re.split('\n') if 'GPU' in i]))
 
-def download(url, 

[GitHub] marcoabreu closed pull request #11151: [MXNET-472] ccache for docker builds

[GitBox]

marcoabreu closed pull request #11151: [MXNET-472] ccache for docker builds
URL: https://github.com/apache/incubator-mxnet/pull/11151
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.gitignore b/.gitignore
index d585672ab7d..416741a5e70 100644
--- a/.gitignore
+++ b/.gitignore
@@ -166,3 +166,7 @@ python/.eggs
 *DartConfiguration.tcl
 tests/Makefile
 tests/mxnet_unit_tests
+
+# generated wrappers for ccache
+cc
+cxx
diff --git a/Makefile b/Makefile
index 03212841fa3..ff4446ab80c 100644
--- a/Makefile
+++ b/Makefile
@@ -477,7 +477,7 @@ endif
 $(PS_PATH)/build/libps.a: PSLITE
 
 PSLITE:
-   $(MAKE) CXX=$(CXX) DEPS_PATH=$(DEPS_PATH) -C $(PS_PATH) ps
+   $(MAKE) CXX="$(CXX)" DEPS_PATH="$(DEPS_PATH)" -C $(PS_PATH) ps
 
 $(DMLC_CORE)/libdmlc.a: DMLCCORE
 
diff --git a/ci/README.md b/ci/README.md
index 1c59a3af7c8..ca46434a30f 100644
--- a/ci/README.md
+++ b/ci/README.md
@@ -54,7 +54,7 @@ The artifacts are located in the build/ directory in the 
project root. In case
 
 ## Add a platform
 
-To add a platform, you should add the appropiate dockerfile in
+To add a platform, you should add the appropriate dockerfile in
 docker/Dockerfile.build. and add a shell function named
 build_ to the file docker/runtime_functions.sh with build
 instructions for that platform.
@@ -63,3 +63,9 @@ instructions for that platform.
 Due to current limitations of the CMake build system creating artifacts in the
 source 3rdparty folder of the parent mxnet sources concurrent builds of
 different platforms is NOT SUPPORTED.
+
+## ccache
+For all builds a directory from the host system is mapped where ccache will 
store cached 
+compiled object files (defaults to /tmp/ci_ccache). This will speed up 
rebuilds 
+significantly. You can set this directory explicitly by setting CCACHE_DIR 
environment 
+variable. All ccache instances are currently set to be 10 Gigabytes max in 
size.
diff --git a/ci/build.py b/ci/build.py
index 4fe76eecf31..c7e84311263 100755
--- a/ci/build.py
+++ b/ci/build.py
@@ -33,13 +33,15 @@
 import shutil
 import subprocess
 import sys
+import tempfile
 from copy import deepcopy
 from itertools import chain
 from subprocess import call, check_call
 from typing import *
 
+CCACHE_MAXSIZE = '10G'
 
-def get_platforms(path: Optional[str]="docker"):
+def get_platforms(path: Optional[str] = "docker"):
 """Get a list of architectures given our dockerfiles"""
 dockerfiles = glob.glob(os.path.join(path, "Dockerfile.build.*"))
 dockerfiles = list(filter(lambda x: x[-1] != '~', dockerfiles))
@@ -72,11 +74,11 @@ def build_docker(platform: str, docker_binary: str, 
registry: str) -> None:
 tag = get_docker_tag(platform=platform, registry=registry)
 logging.info("Building container tagged '%s' with %s", tag, docker_binary)
 cmd = [docker_binary, "build",
-"-f", get_dockerfile(platform),
-"--build-arg", "USER_ID={}".format(os.getuid()),
-"--cache-from", tag,
-"-t", tag,
-"docker"]
+   "-f", get_dockerfile(platform),
+   "--build-arg", "USER_ID={}".format(os.getuid()),
+   "--cache-from", tag,
+   "-t", tag,
+   "docker"]
 logging.info("Running command: '%s'", ' '.join(cmd))
 check_call(cmd)
 
@@ -102,8 +104,10 @@ def _get_local_image_id(docker_binary, docker_tag):
 
 def get_mxnet_root() -> str:
 curpath = os.path.abspath(os.path.dirname(__file__))
+
 def is_mxnet_root(path: str) -> bool:
 return os.path.exists(os.path.join(path, ".mxnet_root"))
+
 while not is_mxnet_root(curpath):
 parent = os.path.abspath(os.path.join(curpath, os.pardir))
 if parent == curpath:
@@ -116,10 +120,20 @@ def buildir() -> str:
 return os.path.join(get_mxnet_root(), "build")
 
 
+def default_ccache_dir() -> str:
+if 'CCACHE_DIR' in os.environ:
+ccache_dir = os.path.realpath(os.environ['CCACHE_DIR'])
+os.makedirs(ccache_dir, exist_ok=True)
+return ccache_dirpython
+# Share ccache across containers
+return os.path.join(tempfile.gettempdir(), "ci_ccache")
+
+
 def container_run(platform: str,
   docker_binary: str,
   docker_registry: str,
   shared_memory_size: str,
+  local_ccache_dir: str,
   command: List[str],
   dry_run: bool = False,
   into_container: bool = False) -> str:
@@ -128,12 +142,17 @@ def container_run(platform: str,
 local_build_folder = buildir()
 # We need to create it first, otherwise it will be created by the docker 
daemon with root only permissions
 os.makedirs(local_build_folder, exist_ok=True)
+os.makedirs(local_ccache_dir, exist_ok=True)
+logging.info("Using ccache 

[GitHub] marcoabreu closed pull request #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

marcoabreu closed pull request #11181: [MXNET-525] Add retry logic to download 
functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/mxnet/gluon/utils.py b/python/mxnet/gluon/utils.py
index 818aa3d2a3b..06b91fadcee 100644
--- a/python/mxnet/gluon/utils.py
+++ b/python/mxnet/gluon/utils.py
@@ -171,7 +171,7 @@ def check_sha1(filename, sha1_hash):
 return sha1.hexdigest() == sha1_hash
 
 
-def download(url, path=None, overwrite=False, sha1_hash=None):
+def download(url, path=None, overwrite=False, sha1_hash=None, retries=5):
 """Download an given URL
 
 Parameters
@@ -186,6 +186,8 @@ def download(url, path=None, overwrite=False, 
sha1_hash=None):
 sha1_hash : str, optional
 Expected sha1 hash in hexadecimal digits. Will ignore existing file 
when hash is specified
 but doesn't match.
+retries : integer, default 5
+The number of times to attempt the download in case of failure or non 
200 return codes
 
 Returns
 ---
@@ -200,26 +202,37 @@ def download(url, path=None, overwrite=False, 
sha1_hash=None):
 fname = os.path.join(path, url.split('/')[-1])
 else:
 fname = path
+assert retries >= 0, "Number of retries should be at least 0"
 
 if overwrite or not os.path.exists(fname) or (sha1_hash and not 
check_sha1(fname, sha1_hash)):
 dirname = os.path.dirname(os.path.abspath(os.path.expanduser(fname)))
 if not os.path.exists(dirname):
 os.makedirs(dirname)
-
-print('Downloading %s from %s...'%(fname, url))
-r = requests.get(url, stream=True)
-if r.status_code != 200:
-raise RuntimeError("Failed downloading url %s"%url)
-with open(fname, 'wb') as f:
-for chunk in r.iter_content(chunk_size=1024):
-if chunk: # filter out keep-alive new chunks
-f.write(chunk)
-
-if sha1_hash and not check_sha1(fname, sha1_hash):
-raise UserWarning('File {} is downloaded but the content hash does 
not match. ' \
-  'The repo may be outdated or download may be 
incomplete. ' \
-  'If the "repo_url" is overridden, consider 
switching to ' \
-  'the default repo.'.format(fname))
+while retries+1 > 0:
+# Disable pyling too broad Exception
+# pylint: disable=W0703
+try:
+print('Downloading %s from %s...'%(fname, url))
+r = requests.get(url, stream=True)
+if r.status_code != 200:
+raise RuntimeError("Failed downloading url %s"%url)
+with open(fname, 'wb') as f:
+for chunk in r.iter_content(chunk_size=1024):
+if chunk: # filter out keep-alive new chunks
+f.write(chunk)
+if sha1_hash and not check_sha1(fname, sha1_hash):
+raise UserWarning('File {} is downloaded but the content 
hash does not match.'\
+  ' The repo may be outdated or download 
may be incomplete. '\
+  'If the "repo_url" is overridden, 
consider switching to '\
+  'the default repo.'.format(fname))
+break
+except Exception as e:
+retries -= 1
+if retries <= 0:
+raise e
+else:
+print("download failed, retrying, {} attempt{} left"
+  .format(retries, 's' if retries > 1 else ''))
 
 return fname
 
diff --git a/python/mxnet/test_utils.py b/python/mxnet/test_utils.py
index bcdcc9c6408..686802d3c48 100644
--- a/python/mxnet/test_utils.py
+++ b/python/mxnet/test_utils.py
@@ -1367,7 +1367,7 @@ def list_gpus():
 pass
 return range(len([i for i in re.split('\n') if 'GPU' in i]))
 
-def download(url, fname=None, dirname=None, overwrite=False):
+def download(url, fname=None, dirname=None, overwrite=False, retries=5):
 """Download an given URL
 
 Parameters
@@ -1385,12 +1385,17 @@ def download(url, fname=None, dirname=None, 
overwrite=False):
 Default is false, which means skipping download if the local file
 exists. If true, then download the url to overwrite the local file if
 exists.
+retries : integer, default 5
+The number of times to attempt the download in case of failure or non 
200 return codes
 
 Returns
 ---
 str
 The filename of the downloaded file
 """
+
+assert retries 

[incubator-mxnet] branch master updated: [MXNET-472] ccache for docker builds (#11151)

[marcoabreu]

This is an automated email from the ASF dual-hosted git repository.

marcoabreu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 6e4a50f  [MXNET-472] ccache for docker builds (#11151)
6e4a50f is described below

commit 6e4a50f96ff2da5ec1bf65e5458b60e6a085b17d
Author: Marco de Abreu 
AuthorDate: Fri Jun 8 03:12:30 2018 +0200

[MXNET-472] ccache for docker builds (#11151)

* [MXNET-472] Add ccache support to docker builds

* Added ccache stages to all containers

* Refactored ccache installs in docker images

* Reformatted build.py

* Added ccache install to android docker builds

* Improved setting ccache directory and max cache size

* Added ccache to cmake based docker builds

* Removed unnessesary yum install from centos7 ccache build

* Added default compilers for ccache for docker builds

* Added README comment about ccache mapping into docker builds

* Reverted not working ccache configurations

* Added comments about ccache installation

* Move install scripts

* Update ubuntu_r.sh
---
 .gitignore |   4 +
 Makefile   |   2 +-
 ci/README.md   |   8 +-
 ci/build.py|  61 +++
 ci/docker/Dockerfile.build.android_arm64   |  10 ++
 ci/docker/Dockerfile.build.android_armv7   |  10 ++
 ci/docker/Dockerfile.build.arm64   |  10 ++
 ci/docker/Dockerfile.build.armv6   |  11 +-
 ci/docker/Dockerfile.build.armv7   |  10 ++
 ci/docker/Dockerfile.build.centos7_cpu |   2 +
 ci/docker/Dockerfile.build.centos7_gpu |   2 +
 ci/docker/Dockerfile.build.jetson  |  13 +++
 ci/docker/Dockerfile.build.ubuntu_build_cuda   |   2 +
 ci/docker/Dockerfile.build.ubuntu_cpu  |   2 +
 ci/docker/Dockerfile.build.ubuntu_gpu  |   2 +
 .../centos7_ccache.sh} |  37 ---
 .../ubuntu_ccache.sh}  |  40 +---
 ci/docker/install/ubuntu_r.sh  |   2 +-
 ci/docker/runtime_functions.sh | 111 ++---
 make/config.mk |   6 ++
 make/crosscompile.jetson.mk|   4 +-
 21 files changed, 285 insertions(+), 64 deletions(-)

diff --git a/.gitignore b/.gitignore
index d585672..416741a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -166,3 +166,7 @@ python/.eggs
 *DartConfiguration.tcl
 tests/Makefile
 tests/mxnet_unit_tests
+
+# generated wrappers for ccache
+cc
+cxx
diff --git a/Makefile b/Makefile
index 0321284..ff4446a 100644
--- a/Makefile
+++ b/Makefile
@@ -477,7 +477,7 @@ endif
 $(PS_PATH)/build/libps.a: PSLITE
 
 PSLITE:
-   $(MAKE) CXX=$(CXX) DEPS_PATH=$(DEPS_PATH) -C $(PS_PATH) ps
+   $(MAKE) CXX="$(CXX)" DEPS_PATH="$(DEPS_PATH)" -C $(PS_PATH) ps
 
 $(DMLC_CORE)/libdmlc.a: DMLCCORE
 
diff --git a/ci/README.md b/ci/README.md
index 1c59a3a..ca46434 100644
--- a/ci/README.md
+++ b/ci/README.md
@@ -54,7 +54,7 @@ The artifacts are located in the build/ directory in the 
project root. In case
 
 ## Add a platform
 
-To add a platform, you should add the appropiate dockerfile in
+To add a platform, you should add the appropriate dockerfile in
 docker/Dockerfile.build. and add a shell function named
 build_ to the file docker/runtime_functions.sh with build
 instructions for that platform.
@@ -63,3 +63,9 @@ instructions for that platform.
 Due to current limitations of the CMake build system creating artifacts in the
 source 3rdparty folder of the parent mxnet sources concurrent builds of
 different platforms is NOT SUPPORTED.
+
+## ccache
+For all builds a directory from the host system is mapped where ccache will 
store cached 
+compiled object files (defaults to /tmp/ci_ccache). This will speed up 
rebuilds 
+significantly. You can set this directory explicitly by setting CCACHE_DIR 
environment 
+variable. All ccache instances are currently set to be 10 Gigabytes max in 
size.
diff --git a/ci/build.py b/ci/build.py
index 4fe76ee..c7e8431 100755
--- a/ci/build.py
+++ b/ci/build.py
@@ -33,13 +33,15 @@ import re
 import shutil
 import subprocess
 import sys
+import tempfile
 from copy import deepcopy
 from itertools import chain
 from subprocess import call, check_call
 from typing import *
 
+CCACHE_MAXSIZE = '10G'
 
-def get_platforms(path: Optional[str]="docker"):
+def get_platforms(path: Optional[str] = "docker"):
 """Get a list of architectures given our dockerfiles"""
 dockerfiles = glob.glob(os.path.join(path, "Dockerfile.build.*"))
 dockerfiles = list(filter(lambda x: x[-1] != '~', dockerfiles))
@@ -72,11 +74,11 @@ def 

[GitHub] marcoabreu commented on issue #11176: Failed to find any forward convolution algorithm.

[GitBox]

marcoabreu commented on issue #11176: Failed to find any forward convolution 
algorithm.
URL: 
https://github.com/apache/incubator-mxnet/issues/11176#issuecomment-395613158
 
 
   This is not CI related


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on issue #11076: [MXNET-491] Use depthwise convolution by cuDNNv7 if available, updated version

[GitBox]

haojin2 commented on issue #11076: [MXNET-491] Use depthwise convolution by 
cuDNNv7 if available, updated version
URL: https://github.com/apache/incubator-mxnet/pull/11076#issuecomment-395612503
 
 
   @BiranLi Can you share some more details on how you're doing this benchmark? 
Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

marcoabreu commented on a change in pull request #11147: changes made to 
publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193928415
 
 

 ##
 File path: scala-package/dev/compile-mxnet-backend.sh
 ##
 @@ -0,0 +1,107 @@
+#!/bin/bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# (Yizhi) This is mainly inspired by the script in apache/spark.
+# I did some modificaiton to get it with our project.
+#
+
+set -e
+echo "Compiling MXNet Backend, Hang tight!."
+
+if [[ ($# -ne 3) || ( $1 == "--help") ||  $1 == "-h" ]]; then
+  echo "Usage: [-h|--help]   
" 1>&2
+  exit 1
+fi
+PLATFORM=$1
+MXNETDIR=$2
+
+
+# below routine shamelessly copied from
+# 
https://github.com/apache/incubator-mxnet/blob/master/setup-utils/install-mxnet-osx-python.sh
+
+chkret() {
+   cmd=$*
+   echo "$cmd"
+   $cmd
+   ret=$?
+   if [[ ${ret} != 0 ]]; then
+   echo " "
+   echo "ERROR: Return value non-zero for: $cmd"
+   echo " "
+   exit 1
+   fi
+} # chkret()
+
+UNAME=`uname -s`
+chkret pushd $MXNETDIR
+chkret git submodule update --init --recursive
+
+# don't want to overwrite an existing config file
+cp make/config.mk ./config.mk
+
+if [[ $PLATFORM == "osx-x86_64-cpu" ]];
+then
+echo "Building MXNet Backend on MAC OS"
+echo "ADD_CFLAGS += -I/usr/local/opt/opencv@2/include" >> ./config.mk
 
 Review comment:
   During release?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11147: changes made to publish to maven

[GitBox]

lanking520 commented on issue #11147: changes made to publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#issuecomment-395611876
 
 
   @marcoabreu this one https://github.com/apache/incubator-mxnet/issues/11193


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on issue #11113: Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on CPU

[GitBox]

haojin2 commented on issue #3: Support for dot(dns, csr) = dns and dot(dns, 
csr.T) = dns on CPU
URL: https://github.com/apache/incubator-mxnet/pull/3#issuecomment-395611730
 
 
   LGTM, will wait for @eric-haibin-lin to take a final look.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin opened a new pull request #11194: [WIP] Enable CUDNN for conv1D

[GitBox]

eric-haibin-lin opened a new pull request #11194: [WIP] Enable CUDNN for conv1D
URL: https://github.com/apache/incubator-mxnet/pull/11194
 
 
   ## Description ##
   
   
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] XiaotaoChen commented on issue #11113: Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on CPU

[GitBox]

XiaotaoChen commented on issue #3: Support for dot(dns, csr) = dns and 
dot(dns, csr.T) = dns on CPU
URL: https://github.com/apache/incubator-mxnet/pull/3#issuecomment-395611446
 
 
   @haojin2 @eric-haibin-lin  I have solved the problems mentioned in your 
comments. Can you accept this PR ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham commented on issue #11191: [MXNET-530] Remove install page artifacts

[GitBox]

aaronmarkham commented on issue #11191: [MXNET-530] Remove install page 
artifacts
URL: https://github.com/apache/incubator-mxnet/pull/11191#issuecomment-395610142
 
 
   We need this so the new version selector for install doesn't get 
overwritten! Looks good!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev commented on issue #11189: Adding support for dict in infer_shape

[GitBox]

spidyDev commented on issue #11189: Adding support for dict in  infer_shape
URL: https://github.com/apache/incubator-mxnet/pull/11189#issuecomment-395609845
 
 
   @piiswrong  Thanks. That should do the trick. I agree no need to change the 
interface.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev closed pull request #11189: Adding support for dict in infer_shape

[GitBox]

spidyDev closed pull request #11189: Adding support for dict in  infer_shape
URL: https://github.com/apache/incubator-mxnet/pull/11189
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/mxnet/symbol/symbol.py b/python/mxnet/symbol/symbol.py
index fc1a71c203b..68b1891ad7f 100644
--- a/python/mxnet/symbol/symbol.py
+++ b/python/mxnet/symbol/symbol.py
@@ -1068,7 +1068,7 @@ def infer_shape_partial(self, *args, **kwargs):
 
 def _infer_shape_impl(self, partial, *args, **kwargs):
 """The actual implementation for calling shape inference API."""
-# pylint: disable=too-many-locals
+# pylint: disable=too-many-locals, too-many-nested-blocks
 if len(args) != 0 and len(kwargs) != 0:
 raise ValueError('Can only specify known argument \
 shapes either by positional or kwargs way.')
@@ -1078,11 +1078,25 @@ def _infer_shape_impl(self, partial, *args, **kwargs):
 keys = c_array(ctypes.c_char_p, [])
 for i, s in enumerate(args):
 if s is not None:
-if not isinstance(s, tuple):
-raise TypeError("Arguments need to be shapes (tuple), "
-"but argument %d is %s." % (i, 
type(s)))
-sdata.extend(s)
-indptr.append(len(sdata))
+if isinstance(s, dict):
+str_keys = []
+for k, v in s.items():
+if not isinstance(v, tuple):
+raise TypeError("Arguments need to be shapes 
(tuple), "
+"but '%s' is %s." % (k, 
type(v)))
+if not isinstance(k, string_types):
+raise TypeError("Key should be of string type 
but is %s"
+% (type(k)))
+str_keys.append(k)
+sdata.extend(v)
+indptr.append(len(sdata))
+keys = c_str_array(str_keys)
+else:
+if not isinstance(s, tuple):
+raise TypeError("Arguments need to be shapes 
(tuple), "
+"but argument %d is %s." % (i, 
type(s)))
+sdata.extend(s)
+indptr.append(len(sdata))
 else:
 str_keys = []
 for k, v in kwargs.items():
@@ -1093,6 +1107,7 @@ def _infer_shape_impl(self, partial, *args, **kwargs):
 sdata.extend(v)
 indptr.append(len(sdata))
 keys = c_str_array(str_keys)
+
 arg_shape_size = mx_uint()
 arg_shape_ndim = ctypes.POINTER(mx_uint)()
 arg_shape_data = ctypes.POINTER(ctypes.POINTER(mx_uint))()


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on a change in pull request #11179: [MXNET-404] elemwise_add/sub between rsp and rsp on GPU

[GitBox]

haojin2 commented on a change in pull request #11179: [MXNET-404] 
elemwise_add/sub between rsp and rsp on GPU
URL: https://github.com/apache/incubator-mxnet/pull/11179#discussion_r193922440
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_op_basic.cu
 ##
 @@ -22,12 +22,141 @@
  * \file elemwise_binary_scalar_op.cu
  * \brief GPU Implementation of unary function.
  */
+#include 
 #include "./elemwise_binary_op.h"
 #include "./elemwise_binary_op-inl.h"
 
 namespace mxnet {
 namespace op {
 
+template
+struct RspElemwiseKernel {
+  template
+  static MSHADOW_XINLINE void Map(int i, DType* out, const IType* lookup_table,
+  const DType* data, const IType* indices,
+  const nnvm::dim_t nz_rows, const nnvm::dim_t 
num_cols) {
+if (i < nz_rows * num_cols) {
+  const nnvm::dim_t row = i / num_cols;
+  const nnvm::dim_t col = i % num_cols;
+  const nnvm::dim_t out_row = lookup_table[indices[row]] - 1;
+  const nnvm::dim_t out_idx = out_row * num_cols + col;
+  out[out_idx] = OP::Map(out[out_idx], data[i]);
+}
+  }
+};
+
+template
+void ElemwiseBinaryOp::RspRspOp(mshadow::Stream *s,
 
 Review comment:
   BTW correctness is double-checked in benchmark script during the warmup.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on a change in pull request #11179: [MXNET-404] elemwise_add/sub between rsp and rsp on GPU

[GitBox]

haojin2 commented on a change in pull request #11179: [MXNET-404] 
elemwise_add/sub between rsp and rsp on GPU
URL: https://github.com/apache/incubator-mxnet/pull/11179#discussion_r193922389
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_op_basic.cu
 ##
 @@ -22,12 +22,141 @@
  * \file elemwise_binary_scalar_op.cu
  * \brief GPU Implementation of unary function.
  */
+#include 
 #include "./elemwise_binary_op.h"
 #include "./elemwise_binary_op-inl.h"
 
 namespace mxnet {
 namespace op {
 
+template
+struct RspElemwiseKernel {
+  template
+  static MSHADOW_XINLINE void Map(int i, DType* out, const IType* lookup_table,
+  const DType* data, const IType* indices,
+  const nnvm::dim_t nz_rows, const nnvm::dim_t 
num_cols) {
+if (i < nz_rows * num_cols) {
+  const nnvm::dim_t row = i / num_cols;
+  const nnvm::dim_t col = i % num_cols;
+  const nnvm::dim_t out_row = lookup_table[indices[row]] - 1;
+  const nnvm::dim_t out_idx = out_row * num_cols + col;
+  out[out_idx] = OP::Map(out[out_idx], data[i]);
+}
+  }
+};
+
+template
+void ElemwiseBinaryOp::RspRspOp(mshadow::Stream *s,
 
 Review comment:
   In-place case shares the same code as in-place case between dns and rsp, 
which already has a unit test.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #11189: Adding support for dict in infer_shape

[GitBox]

piiswrong commented on issue #11189: Adding support for dict in  infer_shape
URL: https://github.com/apache/incubator-mxnet/pull/11189#issuecomment-395603033
 
 
   You can use **kwargs for this. No need to change the interface
   
   >>> a = {'a':1, "a-b":2}
   >>> def xx(**kwargs):
   ... print kwargs
   ... 
   >>> xx(**a)
   {'a': 1, 'a-b': 2}


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #11189: Adding support for dict in infer_shape

[GitBox]

piiswrong commented on issue #11189: Adding support for dict in  infer_shape
URL: https://github.com/apache/incubator-mxnet/pull/11189#issuecomment-395603033
 
 
   You can use **kwargs for this. No need to change the interface
   ```
   >>> a = {'a':1, "a-b":2}
   >>> def xx(**kwargs):
   ... print kwargs
   ... 
   >>> xx(**a)
   {'a': 1, 'a-b': 2}
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #11189: Adding support for dict in infer_shape

[GitBox]

piiswrong commented on issue #11189: Adding support for dict in  infer_shape
URL: https://github.com/apache/incubator-mxnet/pull/11189#issuecomment-395603033
 
 
   You can use **kwargs for this. No need to change the interface


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet-site] branch asf-site updated: Nightly build

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 04cdae0  Nightly build
04cdae0 is described below

commit 04cdae017951f4a43bdb17732904131d8851fc04
Author: mxnet-ci 
AuthorDate: Thu Jun 7 23:52:55 2018 +

Nightly build
---
 date.txt | 1 -
 1 file changed, 1 deletion(-)

diff --git a/date.txt b/date.txt
deleted file mode 100644
index f0a6afb..000
--- a/date.txt
+++ /dev/null
@@ -1 +0,0 @@
-Thu Jun  7 23:45:51 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 4743136  Bump the publish timestamp.
4743136 is described below

commit 47431368060a5530d3bcb0b2438207023ad71297
Author: mxnet-ci 
AuthorDate: Thu Jun 7 23:52:59 2018 +

Bump the publish timestamp.
---
 date.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/date.txt b/date.txt
new file mode 100644
index 000..a90e8fc
--- /dev/null
+++ b/date.txt
@@ -0,0 +1 @@
+Thu Jun  7 23:52:59 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 8e8c2a4  Bump the publish timestamp.
8e8c2a4 is described below

commit 8e8c2a445e556aa07934bc256b72e0466ae8e31d
Author: mxnet-ci 
AuthorDate: Thu Jun 7 23:45:51 2018 +

Bump the publish timestamp.
---
 date.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/date.txt b/date.txt
new file mode 100644
index 000..f0a6afb
--- /dev/null
+++ b/date.txt
@@ -0,0 +1 @@
+Thu Jun  7 23:45:51 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] zhreshold closed pull request #11162: Add valid_thresh to contrib.box_nms

[GitBox]

zhreshold closed pull request #11162: Add valid_thresh to contrib.box_nms
URL: https://github.com/apache/incubator-mxnet/pull/11162
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/operator/contrib/bounding_box-inl.cuh 
b/src/operator/contrib/bounding_box-inl.cuh
new file mode 100644
index 000..fb1dacc11f4
--- /dev/null
+++ b/src/operator/contrib/bounding_box-inl.cuh
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file bounding_box-inl.cuh
+ * \brief bounding box CUDA operators
+ * \author Joshua Zhang
+*/
+#ifndef MXNET_OPERATOR_CONTRIB_BOUNDING_BOX_INL_CUH_
+#define MXNET_OPERATOR_CONTRIB_BOUNDING_BOX_INL_CUH_
+#include 
+#include 
+#include 
+#include "../mshadow_op.h"
+#include "../mxnet_op.h"
+#include "../operator_common.h"
+
+namespace mxnet {
+namespace op {
+
+template
+struct valid_score {
+  DType thresh;
+  explicit valid_score(DType _thresh) : thresh(_thresh) {}
+  __host__ __device__ bool operator()(const DType x) {
+return x > thresh;
+  }
+};
+
+template
+int FilterScores(mshadow::Tensor out_scores,
+ mshadow::Tensor out_sorted_index,
+ mshadow::Tensor scores,
+ mshadow::Tensor sorted_index,
+ float valid_thresh) {
+  valid_score pred(static_cast(valid_thresh));
+  DType * end_scores = thrust::copy_if(thrust::device, scores.dptr_, 
scores.dptr_ + scores.MSize(),
+   out_scores.dptr_, pred);
+  thrust::copy_if(thrust::device, sorted_index.dptr_, sorted_index.dptr_ + 
sorted_index.MSize(),
+  scores.dptr_, out_sorted_index.dptr_, pred);
+  return end_scores - out_scores.dptr_;
+}
+
+}  // namespace op
+}  // namespace mxnet
+
+#endif  // MXNET_OPERATOR_CONTRIB_BOUNDING_BOX_INL_CUH_
diff --git a/src/operator/contrib/bounding_box-inl.h 
b/src/operator/contrib/bounding_box-inl.h
index 40dbdd81669..f739dbc8a52 100644
--- a/src/operator/contrib/bounding_box-inl.h
+++ b/src/operator/contrib/bounding_box-inl.h
@@ -49,6 +49,7 @@ enum BoxNMSOpResource {kTempSpace};
 
 struct BoxNMSParam : public dmlc::Parameter {
   float overlap_thresh;
+  float valid_thresh;
   int topk;
   int coord_start;
   int score_index;
@@ -59,6 +60,8 @@ struct BoxNMSParam : public dmlc::Parameter {
   DMLC_DECLARE_PARAMETER(BoxNMSParam) {
 DMLC_DECLARE_FIELD(overlap_thresh).set_default(0.5)
 .describe("Overlapping(IoU) threshold to suppress object with smaller 
score.");
+DMLC_DECLARE_FIELD(valid_thresh).set_default(0)
+.describe("Filter input boxes to those whose scores greater than 
valid_thresh.");
 DMLC_DECLARE_FIELD(topk).set_default(-1)
 .describe("Apply nms to topk boxes with descending scores, -1 to no 
restriction.");
 DMLC_DECLARE_FIELD(coord_start).set_default(2)
@@ -145,6 +148,33 @@ inline uint32_t BoxNMSNumVisibleOutputs(const NodeAttrs& 
attrs) {
   return static_cast(1);
 }
 
+template
+int FilterScores(mshadow::Tensor out_scores,
+ mshadow::Tensor out_sorted_index,
+ mshadow::Tensor scores,
+ mshadow::Tensor sorted_index,
+ float valid_thresh) {
+  index_t j = 0;
+  for (index_t i = 0; i < scores.size(0); i++) {
+if (scores[i] > valid_thresh) {
+  out_scores[j] = scores[i];
+  out_sorted_index[j] = sorted_index[i];
+  j++;
+}
+  }
+  return j;
+}
+
+namespace mshadow_op {
+struct less_than : public mxnet_op::tunable {
+  // a is x, b is sigma
+  template
+  MSHADOW_XINLINE static DType Map(DType a, DType b) {
+return static_cast(a < b);
+  }
+};  // struct equal_to
+}   // namespace mshadow_op
+
 struct corner_to_center {
   template
   MSHADOW_XINLINE static void Map(int i, DType *data, int stride) {
@@ -198,15 +228,28 @@ MSHADOW_XINLINE DType BoxArea(const DType *box, int 
encode) {
   }
 }
 
-// compute areas specialized for nms to reduce computation
+/*!
+ * \brief compute areas specialized for nms to reduce computation
+ * 
+ * 

[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 4a9ed93  Bump the publish timestamp.
4a9ed93 is described below

commit 4a9ed93568677690bbf462b80f5721c987d347dd
Author: mxnet-ci 
AuthorDate: Thu Jun 7 23:29:15 2018 +

Bump the publish timestamp.
---
 date.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/date.txt b/date.txt
new file mode 100644
index 000..edc17e5
--- /dev/null
+++ b/date.txt
@@ -0,0 +1 @@
+Thu Jun  7 23:29:15 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] megaSpoon commented on issue #6418: Performance of deep speech is bad

[GitBox]

megaSpoon commented on issue #6418: Performance of deep speech is bad
URL: 
https://github.com/apache/incubator-mxnet/issues/6418#issuecomment-395591997
 
 
   @Soonhwan-Kwon Could you do me a favor to give me guidance of drawing 
performance curve like you did?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch v1.2.0 updated: changes made to publish to maven (#11147)

[marcoabreu]

This is an automated email from the ASF dual-hosted git repository.

marcoabreu pushed a commit to branch v1.2.0
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/v1.2.0 by this push:
 new 60dff17  changes made to publish to maven (#11147)
60dff17 is described below

commit 60dff17e6a4e7d9c8f05acba4502c42b65f21330
Author: Naveen Swamy 
AuthorDate: Thu Jun 7 15:53:23 2018 -0700

changes made to publish to maven (#11147)

* changes made to publish to maven

* update VERSION to 1.2.1-SNAPSHOT

* compile backend only when passed flag

* remove the compile.backend flag
---
 Makefile|  14 ++--
 scala-package/assembly/linux-x86_64-cpu/pom.xml |  39 +++--
 scala-package/assembly/linux-x86_64-gpu/pom.xml |  36 +++-
 scala-package/assembly/osx-x86_64-cpu/pom.xml   |  36 +++-
 scala-package/assembly/pom.xml  |   2 +-
 scala-package/core/pom.xml  |   6 +-
 scala-package/dev/compile-mxnet-backend.sh  | 107 
 scala-package/examples/pom.xml  |   6 +-
 scala-package/infer/pom.xml |   4 +-
 scala-package/init-native/linux-x86_64/pom.xml  |   4 +-
 scala-package/init-native/osx-x86_64/pom.xml|   4 +-
 scala-package/init-native/pom.xml   |   2 +-
 scala-package/init/pom.xml  |  50 +++
 scala-package/macros/pom.xml|   6 +-
 scala-package/native/linux-x86_64-cpu/pom.xml   |   4 +-
 scala-package/native/linux-x86_64-gpu/pom.xml   |   4 +-
 scala-package/native/osx-x86_64-cpu/pom.xml |   4 +-
 scala-package/native/pom.xml|   2 +-
 scala-package/pom.xml   |  13 ++-
 scala-package/spark/pom.xml |   4 +-
 20 files changed, 282 insertions(+), 65 deletions(-)

diff --git a/Makefile b/Makefile
index 3317fc5..4db7319 100644
--- a/Makefile
+++ b/Makefile
@@ -576,6 +576,7 @@ scalaclean:
 scalapkg:
(cd $(ROOTDIR)/scala-package; \
mvn package -P$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) 
-Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dcurrent_libdir="$(ROOTDIR)/lib" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a")
@@ -583,12 +584,14 @@ scalapkg:
 scalatest:
(cd $(ROOTDIR)/scala-package; \
mvn verify -P$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) 
-Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a" 
$(SCALA_TEST_ARGS))
 
 scalainstall:
(cd $(ROOTDIR)/scala-package; \
mvn install -P$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) 
-DskipTests -Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a")
 
@@ -596,23 +599,24 @@ scalarelease-dryrun:
(cd $(ROOTDIR)/scala-package; \
mvn release:clean release:prepare -DdryRun=true 
-DautoVersionSubmodules=true \
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \
-   -Darguments=""-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ 
-Dcxx=\""$(CXX)\""\ -Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
+   -Darguments=""-Dbuild\.platform=\""$(SCALA_PKG_PROFILE)\""\ 
-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ -Dcxx=\""$(CXX)\""\ 
-Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
 
-scalarelease:
+scalarelease-prepare:
(cd $(ROOTDIR)/scala-package; \
mvn release:clean release:prepare -DautoVersionSubmodules=true \
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \
-   -Darguments=""-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ 
-Dcxx=\""$(CXX)\""\ -Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
+   -Darguments=""-Dbuild\.platform=\""$(SCALA_PKG_PROFILE)\""\ 
-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ -Dcxx=\""$(CXX)\""\ 
-Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
 
 scalarelease-perform:
(cd $(ROOTDIR)/scala-package; \
-   mvn release:clean release:prepare -DautoVersionSubmodules=true \
+   mvn release:perform -DautoVersionSubmodules=true \
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \
-   -Darguments=""-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ 
-Dcxx=\""$(CXX)\""\ -Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 

[GitHub] marcoabreu closed pull request #11147: changes made to publish to maven

[GitBox]

marcoabreu closed pull request #11147: changes made to publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/Makefile b/Makefile
index 3317fc53b11..4db73195a6f 100644
--- a/Makefile
+++ b/Makefile
@@ -576,6 +576,7 @@ scalaclean:
 scalapkg:
(cd $(ROOTDIR)/scala-package; \
mvn package -P$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) 
-Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dcurrent_libdir="$(ROOTDIR)/lib" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a")
@@ -583,12 +584,14 @@ scalapkg:
 scalatest:
(cd $(ROOTDIR)/scala-package; \
mvn verify -P$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) 
-Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a" 
$(SCALA_TEST_ARGS))
 
 scalainstall:
(cd $(ROOTDIR)/scala-package; \
mvn install -P$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) 
-DskipTests -Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a")
 
@@ -596,23 +599,24 @@ scalarelease-dryrun:
(cd $(ROOTDIR)/scala-package; \
mvn release:clean release:prepare -DdryRun=true 
-DautoVersionSubmodules=true \
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \
-   -Darguments=""-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ 
-Dcxx=\""$(CXX)\""\ -Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
+   -Darguments=""-Dbuild\.platform=\""$(SCALA_PKG_PROFILE)\""\ 
-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ -Dcxx=\""$(CXX)\""\ 
-Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
 
-scalarelease:
+scalarelease-prepare:
(cd $(ROOTDIR)/scala-package; \
mvn release:clean release:prepare -DautoVersionSubmodules=true \
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \
-   -Darguments=""-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ 
-Dcxx=\""$(CXX)\""\ -Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
+   -Darguments=""-Dbuild\.platform=\""$(SCALA_PKG_PROFILE)\""\ 
-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ -Dcxx=\""$(CXX)\""\ 
-Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
 
 scalarelease-perform:
(cd $(ROOTDIR)/scala-package; \
-   mvn release:clean release:prepare -DautoVersionSubmodules=true \
+   mvn release:perform -DautoVersionSubmodules=true \
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \
-   -Darguments=""-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ 
-Dcxx=\""$(CXX)\""\ -Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
+   -Darguments=""-Dbuild\.platform=\""$(SCALA_PKG_PROFILE)\""\ 
-DskipTests\ -Dcflags=\""$(CFLAGS)\""\ -Dcxx=\""$(CXX)\""\ 
-Dldflags=\""$(LDFLAGS)\""\ -Dlddeps=\""$(LIB_DEP) 
$(ROOTDIR)/lib/libmxnet.a\)
 
 scaladeploy:
(cd $(ROOTDIR)/scala-package; \
mvn deploy 
-Papache-release,$(SCALA_PKG_PROFILE),$(SCALA_VERSION_PROFILE) \-DskipTests 
-Dcxx="$(CXX)" \
+   -Dbuild.platform="$(SCALA_PKG_PROFILE)" \
-Dcflags="$(CFLAGS)" -Dldflags="$(LDFLAGS)" \
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a")
 
diff --git a/scala-package/assembly/linux-x86_64-cpu/pom.xml 
b/scala-package/assembly/linux-x86_64-cpu/pom.xml
index 869c14cf7cd..5a91566d482 100644
--- a/scala-package/assembly/linux-x86_64-cpu/pom.xml
+++ b/scala-package/assembly/linux-x86_64-cpu/pom.xml
@@ -6,7 +6,7 @@
   
 org.apache.mxnet
 mxnet-full-parent_2.11
-1.2.0-SNAPSHOT
+1.2.1-SNAPSHOT
 ../pom.xml
   
 
@@ -18,36 +18,57 @@
 
   org.apache.mxnet
   mxnet-core_${scala.binary.version}
-  1.2.0-SNAPSHOT
+  1.2.1-SNAPSHOT
 
 
   org.apache.mxnet
   libmxnet-scala-linux-x86_64-cpu
-  1.2.0-SNAPSHOT
+  1.2.1-SNAPSHOT
   so
 
 
   org.apache.mxnet
   mxnet-infer_${scala.binary.version}
-  1.2.0-SNAPSHOT
+  1.2.1-SNAPSHOT
 
   
 
   
 
   
-org.apache.maven.plugins
-maven-deploy-plugin
-false
+

[GitHub] marcoabreu commented on issue #10827: [MXNET-405][WIP] Add 2 new pipelines to the Official CI and run nightly tests.

[GitBox]

marcoabreu commented on issue #10827: [MXNET-405][WIP] Add 2 new pipelines to 
the Official CI and run nightly tests. 
URL: https://github.com/apache/incubator-mxnet/pull/10827#issuecomment-395590433
 
 
   Could you get the comments addressed by today so I can review everything 
again?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

nswamy commented on a change in pull request #11147: changes made to publish to 
maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193911274
 
 

 ##
 File path: scala-package/pom.xml
 ##
 @@ -30,12 +30,13 @@
 scm:git:g...@github.com:apache/incubator-mxnet.git
 
scm:git:g...@github.com:apache/incubator-mxnet.git
 https://github.com/apache/incubator-mxnet
-mxnet-parent_2.11-1.2.0
+HEAD
 
 Review comment:
   Its just telling what tagName should be used when releasing. when HEAD it 
uses the name of the artifact defined in the pom file + version. We are not 
using this anyway, like I said on the email I have disabled tagging on the 
remote repo


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #10827: [MXNET-405][WIP] Add 2 new pipelines to the Official CI and run nightly tests.

[GitBox]

marcoabreu commented on a change in pull request #10827: [MXNET-405][WIP] Add 2 
new pipelines to the Official CI and run nightly tests. 
URL: https://github.com/apache/incubator-mxnet/pull/10827#discussion_r193911055
 
 

 ##
 File path: ci/docker/Dockerfile.build.ubuntu_emscripten
 ##
 @@ -0,0 +1,58 @@
+# -*- mode: dockerfile -*-
 
 Review comment:
   It's for Amalgamation (JavaScript generation) and we have to keep it. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #10827: [MXNET-405][WIP] Add 2 new pipelines to the Official CI and run nightly tests.

[GitBox]

marcoabreu commented on a change in pull request #10827: [MXNET-405][WIP] Add 2 
new pipelines to the Official CI and run nightly tests. 
URL: https://github.com/apache/incubator-mxnet/pull/10827#discussion_r193911055
 
 

 ##
 File path: ci/docker/Dockerfile.build.ubuntu_emscripten
 ##
 @@ -0,0 +1,58 @@
+# -*- mode: dockerfile -*-
 
 Review comment:
   It's for Amalgamation and we have to keep it. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch devel-android updated: Add android64 support and fix build.py (#11188)

[marcoabreu]

This is an automated email from the ASF dual-hosted git repository.

marcoabreu pushed a commit to branch devel-android
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/devel-android by this push:
 new 14e40b9  Add android64 support and fix build.py (#11188)
14e40b9 is described below

commit 14e40b9b4d8d63b7f98e994c5587d3d47fc7710b
Author: Pedro Larroy <928489+lar...@users.noreply.github.com>
AuthorDate: Fri Jun 8 00:49:36 2018 +0200

Add android64 support and fix build.py (#11188)

* Fix ci/build.py using remote cache locally

* Fixes sources for android64
Add initial android64 build logic.
Remove pthread when linking MXNet in Android. Bionic provides built-in 
support for threads.
---
 CMakeLists.txt   |  4 ++--
 Jenkinsfile  | 10 +
 ci/build.py  | 36 +++-
 ci/docker/Dockerfile.build.android_arm64 |  1 +
 ci/docker/install/android_arm64_ndk.sh   |  5 +++--
 ci/docker/runtime_functions.sh   | 30 +-
 src/operator/random/shuffle_op.cc|  5 +++--
 7 files changed, 69 insertions(+), 22 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index e57c00b..764e8cc 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -15,7 +15,7 @@ mxnet_option(USE_NCCL "Use NVidia NCCL with CUDA" 
OFF)
 mxnet_option(USE_OPENCV   "Build with OpenCV support" ON)
 mxnet_option(USE_OPENMP   "Build with Openmp support" ON)
 mxnet_option(USE_CUDNN"Build with cudnn support"  ON) # one could 
set CUDNN_ROOT for search path
-mxnet_option(USE_SSE  "Build with x86 SSE instruction support" ON)
+mxnet_option(USE_SSE  "Build with x86 SSE instruction support" ON 
IF NOT ARM)
 mxnet_option(USE_F16C "Build with x86 F16C instruction support" 
ON) # autodetects support if ON
 mxnet_option(USE_LAPACK   "Build with lapack support" ON)
 mxnet_option(USE_MKL_IF_AVAILABLE "Use MKL if found" ON)
@@ -360,7 +360,7 @@ if(USE_OPENMP)
   set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} 
${OpenMP_EXE_LINKER_FLAGS}")
 endif()
   endif()
-elseif(UNIX)
+elseif(UNIX AND NOT ANDROID)
   list(APPEND mxnet_LINKER_LIBS pthread)
 endif()
 
diff --git a/Jenkinsfile b/Jenkinsfile
index 28edda0..61b6380 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -458,6 +458,16 @@ try {
   }
 }
   }
+},
+'Android / ARM64':{
+  node('mxnetlinux-cpu') {
+ws('workspace/android64') {
+  timeout(time: max_time, unit: 'MINUTES') {
+init_git()
+docker_run('android_arm64', 'build_android_arm64', false)
+  }
+}
+  }
 }
   } // End of stage('Build')
 
diff --git a/ci/build.py b/ci/build.py
index e52fa79..4fe76ee 100755
--- a/ci/build.py
+++ b/ci/build.py
@@ -221,63 +221,69 @@ def main() -> int:
 help="go in a shell inside the container",
 action='store_true')
 
-parser.add_argument("--docker-registry",
-help="Dockerhub registry name to retrieve cache from",
+parser.add_argument("-d", "--docker-registry",
+help="Dockerhub registry name to retrieve cache from. 
Default is 'mxnetci'",
 default='mxnetci',
 type=str)
 
+parser.add_argument("-c", "--cache", action="store_true",
+help="Enable docker registry cache")
+
 parser.add_argument("command",
 help="command to run in the container",
 nargs='*', action='append', type=str)
 
 args = parser.parse_args()
-docker_registry = args.docker_registry
+def use_cache():
+return args.cache or 'JOB_NAME' in os.environ # we are in Jenkins
+
 command = list(chain(*args.command))
 docker_binary = get_docker_binary(args.nvidiadocker)
 shared_memory_size = args.shared_memory_size
 
-print("into container: {}".format(args.into_container))
 if args.list:
 list_platforms()
 elif args.platform:
 platform = args.platform
-tag = get_docker_tag(platform=platform, registry=docker_registry)
-load_docker_cache(tag=tag, docker_registry=args.docker_registry)
-build_docker(platform, docker_binary, registry=docker_registry)
+tag = get_docker_tag(platform=platform, registry=args.docker_registry)
+if use_cache():
+load_docker_cache(tag=tag, docker_registry=args.docker_registry)
+build_docker(platform, docker_binary, registry=args.docker_registry)
 if args.build_only:
 logging.warning("Container was just built. Exiting due to 
build-only.")
 return 0
 
 if command:
 container_run(platform=platform, docker_binary=docker_binary, 

[GitHub] marcoabreu closed pull request #11188: Add android64 support and fix build.py

[GitBox]

marcoabreu closed pull request #11188: Add android64 support and fix build.py
URL: https://github.com/apache/incubator-mxnet/pull/11188
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/CMakeLists.txt b/CMakeLists.txt
index e57c00b69e9..764e8cc4623 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -15,7 +15,7 @@ mxnet_option(USE_NCCL "Use NVidia NCCL with CUDA" 
OFF)
 mxnet_option(USE_OPENCV   "Build with OpenCV support" ON)
 mxnet_option(USE_OPENMP   "Build with Openmp support" ON)
 mxnet_option(USE_CUDNN"Build with cudnn support"  ON) # one could 
set CUDNN_ROOT for search path
-mxnet_option(USE_SSE  "Build with x86 SSE instruction support" ON)
+mxnet_option(USE_SSE  "Build with x86 SSE instruction support" ON 
IF NOT ARM)
 mxnet_option(USE_F16C "Build with x86 F16C instruction support" 
ON) # autodetects support if ON
 mxnet_option(USE_LAPACK   "Build with lapack support" ON)
 mxnet_option(USE_MKL_IF_AVAILABLE "Use MKL if found" ON)
@@ -360,7 +360,7 @@ if(USE_OPENMP)
   set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} 
${OpenMP_EXE_LINKER_FLAGS}")
 endif()
   endif()
-elseif(UNIX)
+elseif(UNIX AND NOT ANDROID)
   list(APPEND mxnet_LINKER_LIBS pthread)
 endif()
 
diff --git a/Jenkinsfile b/Jenkinsfile
index 28edda00959..61b63807703 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -458,6 +458,16 @@ try {
   }
 }
   }
+},
+'Android / ARM64':{
+  node('mxnetlinux-cpu') {
+ws('workspace/android64') {
+  timeout(time: max_time, unit: 'MINUTES') {
+init_git()
+docker_run('android_arm64', 'build_android_arm64', false)
+  }
+}
+  }
 }
   } // End of stage('Build')
 
diff --git a/ci/build.py b/ci/build.py
index e52fa794bc9..4fe76eecf31 100755
--- a/ci/build.py
+++ b/ci/build.py
@@ -221,63 +221,69 @@ def script_name() -> str:
 help="go in a shell inside the container",
 action='store_true')
 
-parser.add_argument("--docker-registry",
-help="Dockerhub registry name to retrieve cache from",
+parser.add_argument("-d", "--docker-registry",
+help="Dockerhub registry name to retrieve cache from. 
Default is 'mxnetci'",
 default='mxnetci',
 type=str)
 
+parser.add_argument("-c", "--cache", action="store_true",
+help="Enable docker registry cache")
+
 parser.add_argument("command",
 help="command to run in the container",
 nargs='*', action='append', type=str)
 
 args = parser.parse_args()
-docker_registry = args.docker_registry
+def use_cache():
+return args.cache or 'JOB_NAME' in os.environ # we are in Jenkins
+
 command = list(chain(*args.command))
 docker_binary = get_docker_binary(args.nvidiadocker)
 shared_memory_size = args.shared_memory_size
 
-print("into container: {}".format(args.into_container))
 if args.list:
 list_platforms()
 elif args.platform:
 platform = args.platform
-tag = get_docker_tag(platform=platform, registry=docker_registry)
-load_docker_cache(tag=tag, docker_registry=args.docker_registry)
-build_docker(platform, docker_binary, registry=docker_registry)
+tag = get_docker_tag(platform=platform, registry=args.docker_registry)
+if use_cache():
+load_docker_cache(tag=tag, docker_registry=args.docker_registry)
+build_docker(platform, docker_binary, registry=args.docker_registry)
 if args.build_only:
 logging.warning("Container was just built. Exiting due to 
build-only.")
 return 0
 
 if command:
 container_run(platform=platform, docker_binary=docker_binary, 
shared_memory_size=shared_memory_size,
-  command=command, docker_registry=docker_registry)
+  command=command, 
docker_registry=args.docker_registry)
 elif args.print_docker_run:
 print(container_run(platform=platform, 
docker_binary=docker_binary, shared_memory_size=shared_memory_size,
-command=[], dry_run=True, 
docker_registry=docker_registry))
+command=[], dry_run=True, 
docker_registry=args.docker_registry))
 elif args.into_container:
 container_run(platform=platform, docker_binary=docker_binary, 
shared_memory_size=shared_memory_size,
-  command=[], dry_run=False, into_container=True, 
docker_registry=docker_registry)
+  

[GitHub] marcoabreu commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

marcoabreu commented on a change in pull request #11147: changes made to 
publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193910646
 
 

 ##
 File path: scala-package/pom.xml
 ##
 @@ -30,12 +30,13 @@
 scm:git:g...@github.com:apache/incubator-mxnet.git
 
scm:git:g...@github.com:apache/incubator-mxnet.git
 https://github.com/apache/incubator-mxnet
-mxnet-parent_2.11-1.2.0
+HEAD
 
 Review comment:
   I'm not very familiar with that setup, so please bare with me. What exactly 
is this supposed to do? I'd expect it to point to a certain branch, but this 
one would point to the master. What's the use-case here?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

nswamy commented on a change in pull request #11147: changes made to publish to 
maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193909435
 
 

 ##
 File path: scala-package/pom.xml
 ##
 @@ -30,12 +30,13 @@
 scm:git:g...@github.com:apache/incubator-mxnet.git
 
scm:git:g...@github.com:apache/incubator-mxnet.git
 https://github.com/apache/incubator-mxnet
-mxnet-parent_2.11-1.2.0
+HEAD
 
 Review comment:
   Yes, It has to create a new tag every time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

marcoabreu commented on a change in pull request #11147: changes made to 
publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193909260
 
 

 ##
 File path: scala-package/pom.xml
 ##
 @@ -30,12 +30,13 @@
 scm:git:g...@github.com:apache/incubator-mxnet.git
 
scm:git:g...@github.com:apache/incubator-mxnet.git
 https://github.com/apache/incubator-mxnet
-mxnet-parent_2.11-1.2.0
+HEAD
 
 Review comment:
   Is this intended to point to the HEAD (master)?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on issue #11147: changes made to publish to maven

[GitBox]

lanking520 commented on issue #11147: changes made to publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#issuecomment-395588112
 
 
   Thanks for your input, let's keep the issue tracked and solve that. @nswamy  
LGTM + 1


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

marcoabreu commented on a change in pull request #11147: changes made to 
publish to maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193909099
 
 

 ##
 File path: scala-package/dev/compile-mxnet-backend.sh
 ##
 @@ -0,0 +1,107 @@
+#!/bin/bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# (Yizhi) This is mainly inspired by the script in apache/spark.
+# I did some modificaiton to get it with our project.
+#
+
+set -e
+echo "Compiling MXNet Backend, Hang tight!."
+
+if [[ ($# -ne 3) || ( $1 == "--help") ||  $1 == "-h" ]]; then
+  echo "Usage: [-h|--help]   
" 1>&2
+  exit 1
+fi
+PLATFORM=$1
+MXNETDIR=$2
+
+
+# below routine shamelessly copied from
+# 
https://github.com/apache/incubator-mxnet/blob/master/setup-utils/install-mxnet-osx-python.sh
+
+chkret() {
+   cmd=$*
+   echo "$cmd"
+   $cmd
+   ret=$?
+   if [[ ${ret} != 0 ]]; then
+   echo " "
+   echo "ERROR: Return value non-zero for: $cmd"
+   echo " "
+   exit 1
+   fi
+} # chkret()
+
+UNAME=`uname -s`
+chkret pushd $MXNETDIR
+chkret git submodule update --init --recursive
+
+# don't want to overwrite an existing config file
+cp make/config.mk ./config.mk
+
+if [[ $PLATFORM == "osx-x86_64-cpu" ]];
+then
+echo "Building MXNet Backend on MAC OS"
+echo "ADD_CFLAGS += -I/usr/local/opt/opencv@2/include" >> ./config.mk
+echo "ADD_CFLAGS += -I/usr/local/opt/openblas/include" >> ./config.mk
+echo "ADD_LDFLAGS += -L/usr/local/opt/opencv@2/lib" >> ./config.mk
+echo "ADD_LDFLAGS += -L/usr/local/opt/openblas/lib" >> ./config.mk
+echo "USE_OPENMP = 0" >> ./config.mk
+echo "USE_LAPACK_PATH = /usr/local/opt/lapack/lib" >> ./config.mk
+make -j$(sysctl -n hw.ncpu)
+elif [[ $PLATFORM == "linux-x86_64-cpu" ]];
+then
+echo "Building MXNet Backend on Linux CPU"
+echo "ADD_CFLAGS += -I/usr/local/include/opencv" >> ./config.mk
+echo "ADD_LDFLAGS += -L/usr/local/lib" >> ./config.mk
+echo "USE_OPENCV=1" >> ./config.mk
+echo "USE_OPENMP=1" >> ./config.mk
+echo "USE_BLAS=openblas" >> ./config.mk
+echo "USE_LAPACK=1" >> ./config.mk
+echo "USE_DIST_KVSTORE=1" >> ./config.mk
+echo "USE_S3=1" >> ./config.mk
+make -j$(cat /proc/cpuinfo | awk '/^processor/{print $3}' | tail -1)
+elif [[ $PLATFORM == "linux-x86_64-gpu" ]]
+then
+echo "Building MXNet Backend on Linux GPU"
+echo "Building MXNet Backend on Linux CPU"
+echo "ADD_CFLAGS += -I/usr/local/include/opencv" >> ./config.mk
+echo "ADD_LDFLAGS += -L/usr/local/lib" >> ./config.mk
+echo "USE_OPENCV=1" >> ./config.mk
+echo "USE_OPENMP=1" >> ./config.mk
+echo "USE_BLAS=openblas" >> ./config.mk
+echo "USE_LAPACK=1" >> ./config.mk
+echo "USE_DIST_KVSTORE=1" >> ./config.mk
+echo "USE_S3=1" >> ./config.mk
+echo "USE_CUDA=1" >> ./config.mk
+echo "USE_CUDNN=1" >> ./config.mk
+echo "ADD_CFLAGS += -I/usr/local/cuda/include" >> ./config.mk
+echo "ADD_LDFLAGS += -L/usr/local/cuda/lib64/  " >> ./config.mk
+#update th nccl version approriately
+echo "ADD_LDFLAGS += -L/lib/nccl/cuda-9.0/lib  " >> ./config.mk
+eval "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib/nccl/cuda-9.0/lib"
+eval "export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
+make -j$(cat /proc/cpuinfo | awk '/^processor/{print $3}' | tail -1)
+echo "Building MXNet Backend on Linux GPU"
+else
+echo "MY ALIEN OVERLOADS HAVE NOT TOLD WHAT TO DO FOR INVALID INPUT !!!"
 
 Review comment:
   <3


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

nswamy commented on a change in pull request #11147: changes made to publish to 
maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193908873
 
 

 ##
 File path: scala-package/dev/compile-mxnet-backend.sh
 ##
 @@ -0,0 +1,107 @@
+#!/bin/bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# (Yizhi) This is mainly inspired by the script in apache/spark.
+# I did some modificaiton to get it with our project.
+#
+
+set -e
+echo "Compiling MXNet Backend, Hang tight!."
+
+if [[ ($# -ne 3) || ( $1 == "--help") ||  $1 == "-h" ]]; then
+  echo "Usage: [-h|--help]   
" 1>&2
+  exit 1
+fi
+PLATFORM=$1
+MXNETDIR=$2
+
+
+# below routine shamelessly copied from
+# 
https://github.com/apache/incubator-mxnet/blob/master/setup-utils/install-mxnet-osx-python.sh
+
+chkret() {
+   cmd=$*
+   echo "$cmd"
+   $cmd
+   ret=$?
+   if [[ ${ret} != 0 ]]; then
+   echo " "
+   echo "ERROR: Return value non-zero for: $cmd"
+   echo " "
+   exit 1
+   fi
+} # chkret()
+
+UNAME=`uname -s`
+chkret pushd $MXNETDIR
+chkret git submodule update --init --recursive
+
+# don't want to overwrite an existing config file
+cp make/config.mk ./config.mk
+
+if [[ $PLATFORM == "osx-x86_64-cpu" ]];
+then
+echo "Building MXNet Backend on MAC OS"
+echo "ADD_CFLAGS += -I/usr/local/opt/opencv@2/include" >> ./config.mk
 
 Review comment:
   Here is the issue to track 
https://github.com/apache/incubator-mxnet/issues/1119


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy opened a new issue #11193: Fix OpenCV 3.4.1 issue for Scala package for OSX

[GitBox]

nswamy opened a new issue #11193: Fix OpenCV 3.4.1 issue for Scala package for 
OSX
URL: https://github.com/apache/incubator-mxnet/issues/11193
 
 
   Currently OpenCV@2 package along with Scala package seems to be failing due 
to an existing opencv version. 
   modify scala-package/dev/compile-backend.sh to use OpenCV3.4.1
   
   For Q & A and discussion, please start a discussion thread at 
https://discuss.mxnet.io 
   
   ## Description
   (Brief description of the problem in no more than 2 sentences.)
   
   ## Environment info (Required)
   
   ```
   What to do:
   1. Download the diagnosis script from 
https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py
   2. Run the script using `python diagnose.py` and paste its output here.
   
   ```
   
   Package used (Python/R/Scala/Julia):
   (I'm using ...)
   
   For Scala user, please provide:
   1. Java version: (`java -version`)
   2. Maven version: (`mvn -version`)
   3. Scala runtime if applicable: (`scala -version`)
   
   For R user, please provide R `sessionInfo()`:
   
   ## Build info (Required if built from source)
   
   Compiler (gcc/clang/mingw/visual studio):
   
   MXNet commit hash:
   (Paste the output of `git rev-parse HEAD` here.)
   
   Build config:
   (Paste the content of config.mk, or the build command.)
   
   ## Error Message:
   (Paste the complete error message, including stack trace.)
   
   ## Minimum reproducible example
   (If you are using your own code, please provide a short script that 
reproduces the error. Otherwise, please provide link to the existing example.)
   
   ## Steps to reproduce
   (Paste the commands you ran that produced the error.)
   
   1.
   2.
   
   ## What have you tried to solve it?
   
   1.
   2.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #11179: [MXNET-404] elemwise_add/sub between rsp and rsp on GPU

[GitBox]

eric-haibin-lin commented on a change in pull request #11179: [MXNET-404] 
elemwise_add/sub between rsp and rsp on GPU
URL: https://github.com/apache/incubator-mxnet/pull/11179#discussion_r193908123
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_op_basic.cu
 ##
 @@ -22,12 +22,141 @@
  * \file elemwise_binary_scalar_op.cu
  * \brief GPU Implementation of unary function.
  */
+#include 
 #include "./elemwise_binary_op.h"
 #include "./elemwise_binary_op-inl.h"
 
 namespace mxnet {
 namespace op {
 
+template
+struct RspElemwiseKernel {
+  template
+  static MSHADOW_XINLINE void Map(int i, DType* out, const IType* lookup_table,
+  const DType* data, const IType* indices,
+  const nnvm::dim_t nz_rows, const nnvm::dim_t 
num_cols) {
+if (i < nz_rows * num_cols) {
+  const nnvm::dim_t row = i / num_cols;
+  const nnvm::dim_t col = i % num_cols;
+  const nnvm::dim_t out_row = lookup_table[indices[row]] - 1;
+  const nnvm::dim_t out_idx = out_row * num_cols + col;
+  out[out_idx] = OP::Map(out[out_idx], data[i]);
+}
+  }
+};
+
+template
+void ElemwiseBinaryOp::RspRspOp(mshadow::Stream *s,
 
 Review comment:
   do we have unit test for write inplace?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download 
functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#issuecomment-395586083
 
 
   @ThomasDelteil did you see the lint errors?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on a change in pull request #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

ThomasDelteil commented on a change in pull request #11181: [MXNET-525] Add 
retry logic to download functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#discussion_r193908119
 
 

 ##
 File path: python/mxnet/gluon/utils.py
 ##
 @@ -200,26 +202,35 @@ def download(url, path=None, overwrite=False, 
sha1_hash=None):
 fname = os.path.join(path, url.split('/')[-1])
 else:
 fname = path
+assert retries >= 0, "Number of retries should be at least 0"
 
 if overwrite or not os.path.exists(fname) or (sha1_hash and not 
check_sha1(fname, sha1_hash)):
 dirname = os.path.dirname(os.path.abspath(os.path.expanduser(fname)))
 if not os.path.exists(dirname):
 os.makedirs(dirname)
-
-print('Downloading %s from %s...'%(fname, url))
-r = requests.get(url, stream=True)
-if r.status_code != 200:
-raise RuntimeError("Failed downloading url %s"%url)
-with open(fname, 'wb') as f:
-for chunk in r.iter_content(chunk_size=1024):
-if chunk: # filter out keep-alive new chunks
-f.write(chunk)
-
-if sha1_hash and not check_sha1(fname, sha1_hash):
-raise UserWarning('File {} is downloaded but the content hash does 
not match. ' \
-  'The repo may be outdated or download may be 
incomplete. ' \
-  'If the "repo_url" is overridden, consider 
switching to ' \
-  'the default repo.'.format(fname))
+while (retries+1 > 0):
+try:
+print('Downloading %s from %s...'%(fname, url))
+r = requests.get(url, stream=True)
+if r.status_code != 200:
+raise RuntimeError("Failed downloading url %s"%url)
+with open(fname, 'wb') as f:
+for chunk in r.iter_content(chunk_size=1024):
+if chunk: # filter out keep-alive new chunks
+f.write(chunk)
+
+if sha1_hash and not check_sha1(fname, sha1_hash):
+raise UserWarning('File {} is downloaded but the content 
hash does not match. ' \
+  'The repo may be outdated or download 
may be incomplete. ' \
+  'If the "repo_url" is overridden, 
consider switching to ' \
+  'the default repo.'.format(fname))
+break
+except Exception as e:
 
 Review comment:
   I think for this version we can keep it stupid simple. Kellen has a more 
involved version in the works. From the error I have seen due to download, 
ConnectionReset, TLS handshakes aborted,  connection dropped, this should be 
enough. Happy to revisit if it proves incomplete.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download 
functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#issuecomment-395586739
 
 
   Sorry, I meant lint.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed pull request #10435: [MXNET-289] Fix bugs in image classification example

[GitBox]

eric-haibin-lin closed pull request #10435: [MXNET-289] Fix bugs in image 
classification example 
URL: https://github.com/apache/incubator-mxnet/pull/10435
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/example/image-classification/common/fit.py 
b/example/image-classification/common/fit.py
index f5427feae2f..3f37ad3ac59 100755
--- a/example/image-classification/common/fit.py
+++ b/example/image-classification/common/fit.py
@@ -23,13 +23,13 @@
 import math
 import mxnet as mx
 
+def get_epoch_size(args, kv):
+return math.ceil(int(args.num_examples / kv.num_workers) / args.batch_size)
 
 def _get_lr_scheduler(args, kv):
 if 'lr_factor' not in args or args.lr_factor >= 1:
 return (args.lr, None)
-epoch_size = args.num_examples / args.batch_size
-if 'dist' in args.kv_store:
-epoch_size /= kv.num_workers
+epoch_size = get_epoch_size(args, kv)
 begin_epoch = args.load_epoch if args.load_epoch else 0
 if 'pow' in args.lr_step_epochs:
 lr = args.lr
@@ -48,8 +48,10 @@ def _get_lr_scheduler(args, kv):
 
 steps = [epoch_size * (x - begin_epoch)
  for x in step_epochs if x - begin_epoch > 0]
-return (lr, mx.lr_scheduler.MultiFactorScheduler(step=steps, 
factor=args.lr_factor))
-
+if steps:
+return (lr, mx.lr_scheduler.MultiFactorScheduler(step=steps, 
factor=args.lr_factor))
+else:
+return (lr, None)
 
 def _load_model(args, rank=0):
 if 'load_epoch' not in args or args.load_epoch is None:
@@ -153,9 +155,17 @@ def fit(args, network, data_loader, **kwargs):
 head = '%(asctime)-15s Node[' + str(kv.rank) + '] %(message)s'
 logging.basicConfig(level=logging.DEBUG, format=head)
 logging.info('start with arguments %s', args)
+
+epoch_size = get_epoch_size(args, kv)
 
 # data iterators
 (train, val) = data_loader(args, kv)
+if 'dist' in args.kv_store and not 'async' in args.kv_store:
+logging.info('Resizing training data to %d batches per machine', 
epoch_size)
+# resize train iter to ensure each machine has same number of batches 
per epoch
+# if not, dist_sync can hang at the end with one machine waiting for 
other machines
+train = mx.io.ResizeIter(train, epoch_size)
+
 if args.test_io:
 tic = time.time()
 for i, batch in enumerate(train):
@@ -211,11 +221,7 @@ def fit(args, network, data_loader, **kwargs):
 # A limited number of optimizers have a warmup period
 has_warmup = {'lbsgd', 'lbnag'}
 if args.optimizer in has_warmup:
-if 'dist' in args.kv_store:
-nworkers = kv.num_workers
-else:
-nworkers = 1
-epoch_size = args.num_examples / args.batch_size / nworkers
+nworkers = kv.num_workers
 if epoch_size < 1:
 epoch_size = 1
 macrobatch_size = args.macrobatch_size


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on issue #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

ThomasDelteil commented on issue #11181: [MXNET-525] Add retry logic to 
download functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#issuecomment-395586617
 
 
   Yes they've been addressed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed issue #9611: program can't finished normally in dist_sync mode

[GitBox]

eric-haibin-lin closed issue #9611: program can't finished normally in 
dist_sync mode
URL: https://github.com/apache/incubator-mxnet/issues/9611
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download 
functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#issuecomment-395586479
 
 
   
http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/job/PR-11181/1/display/redirect


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10827: [MXNET-405][WIP] Add 2 new pipelines to the Official CI and run nightly tests.

[GitBox]

marcoabreu commented on issue #10827: [MXNET-405][WIP] Add 2 new pipelines to 
the Official CI and run nightly tests. 
URL: https://github.com/apache/incubator-mxnet/pull/10827#issuecomment-395586353
 
 
   Yeah, you don't have to set anything up. I will handle it with my system :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ThomasDelteil commented on issue #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

ThomasDelteil commented on issue #11181: [MXNET-525] Add retry logic to 
download functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#issuecomment-395586192
 
 
   which link?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download functions to fix flaky tests

[GitBox]

marcoabreu commented on issue #11181: [MXNET-525] Add retry logic to download 
functions to fix flaky tests
URL: https://github.com/apache/incubator-mxnet/pull/11181#issuecomment-395586083
 
 
   @ThomasDelteil did you see the link errors?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet-site] branch asf-site updated: Nightly build

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new a478554  Nightly build
a478554 is described below

commit a478554fc22e7933459d24026afb086a4b9f275f
Author: mxnet-ci 
AuthorDate: Thu Jun 7 22:19:21 2018 +

Nightly build
---
 date.txt | 1 -
 1 file changed, 1 deletion(-)

diff --git a/date.txt b/date.txt
deleted file mode 100644
index 8102a20..000
--- a/date.txt
+++ /dev/null
@@ -1 +0,0 @@
-Thu Jun  7 01:37:13 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] kpmurali opened a new pull request #11191: [MXNET-530] Remove install page artifacts

[GitBox]

kpmurali opened a new pull request #11191: [MXNET-530] Remove install page 
artifacts
URL: https://github.com/apache/incubator-mxnet/pull/11191
 
 
   ## Description ##
   Removing the install page artifacts and the changes to incorporate them into 
the build process
   
   ## Checklist ##
   ### Changes ###
   - [ x ] Remove the update_install function
   - [ x ] Remove the references to the above function
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sandeep-krishnamurthy opened a new issue #11192: BatchNorm operator on GPUs are slow with channels_last

[GitBox]

sandeep-krishnamurthy opened a new issue #11192: BatchNorm operator on GPUs are 
slow with channels_last
URL: https://github.com/apache/incubator-mxnet/issues/11192
 
 
   [mx.sym.BatchNorm 
operator](https://mxnet.incubator.apache.org/api/python/symbol/symbol.html#mxnet.symbol.BatchNorm)
 is considerably slower on GPUs with channels_last (axis=-1).
   
   My understanding is that, with channels_first (axis=1) operator calls cuDNN 
implementation and with channels_last(axis=-1) MXNet has its own implementation.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham commented on issue #11128: [MXNET-504] Add version select + queryString capabilities + C++ instructions to install page

[GitBox]

aaronmarkham commented on issue #11128: [MXNET-504] Add version select + 
queryString capabilities + C++ instructions to install page
URL: https://github.com/apache/incubator-mxnet/pull/11128#issuecomment-395576114
 
 
   
![2018-06-07_14-45-38](https://user-images.githubusercontent.com/5974205/41128026-b71e6a64-6a61-11e8-969d-c75135df410f.gif)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kpmurali opened a new pull request #11190: [MXNET-529] Add version select in the API pages

[GitBox]

kpmurali opened a new pull request #11190: [MXNET-529] Add version select in 
the API pages
URL: https://github.com/apache/incubator-mxnet/pull/11190
 
 
   ## Description ##
   Add a version select in the API pages just like the new one for install page 
that loads the appropriate API page.
   
   ## Checklist ##
   ### Changes ###
   - [ x ] Add the version selector feature to API pages
   - [ x ] Add a new JS to provide version navigation
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed pull request #11128: [MXNET-504] Add version select + queryString capabilities + C++ instructions to install page

[GitBox]

szha closed pull request #11128: [MXNET-504] Add version select + queryString 
capabilities + C++ instructions to install page
URL: https://github.com/apache/incubator-mxnet/pull/11128
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/_static/js/options.js b/docs/_static/js/options.js
index 8fe74ee1904..d2d44b07848 100644
--- a/docs/_static/js/options.js
+++ b/docs/_static/js/options.js
@@ -1,10 +1,47 @@
-//$('.burger-link').on('click', function(e) { e.stopPropagation() });
-//$('.burger-link').on('touchstart', function(e) { e.stopPropagation() });
+var versionSelect   = defaultVersion = 'v1.2.0';
+var deviceSelect= 'Linux';
+var languageSelect  = 'Python';
+var processorSelect = 'CPU';
+var environSelect   = 'Pip';
 
 $(document).ready(function () {
 function label(lbl) {
 return lbl.replace(/[ .]/g, '-').toLowerCase();
 }
+
+function setSelects(){
+let urlParams = new URLSearchParams(window.location.search);
+if (urlParams.get('version'))
+versionSelect = urlParams.get('version');
+$('li a:contains(' + versionSelect + 
')').parent().siblings().removeClass('active');
+$('li a:contains(' + versionSelect + ')').parent().addClass('active');
+$('.current-version').html( versionSelect + ' ' );
+if (urlParams.get('device'))
+deviceSelect = urlParams.get('device');
+$('button:contains(' + deviceSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + deviceSelect + ')').addClass('active');
+if (urlParams.get('language'))
+languageSelect = urlParams.get('language');
+$('button:contains(' + languageSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + languageSelect + ')').addClass('active');
+if (urlParams.get('processor'))
+processorSelect = urlParams.get('processor');
+$('button:contains(' + processorSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + processorSelect + ')').addClass('active');
+if (urlParams.get('environ'))
+environSelect = urlParams.get('environ');
+$('button:contains(' + environSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + environSelect + ')').addClass('active');
+showContent();
+if (window.location.href.includes("/install/index.html")) {
+if (versionSelect.includes(defaultVersion)) {
+history.pushState(null, null, '/install/index.html?device=' + 
deviceSelect + '=' + languageSelect + '=' + processorSelect);
+} else {
+history.pushState(null, null, '/install/index.html?version=' + 
versionSelect + '=' + deviceSelect + '=' + languageSelect + 
'=' + processorSelect);
+}
+} 
+}
+
 function showContent() {
 $('.opt-group .opt').each(function(){
 $('.'+label($(this).text())).hide();
@@ -16,11 +53,35 @@ $(document).ready(function () {
 });
 }
 showContent();
+setSelects();
 function setContent() {
 var el = $(this);
+let urlParams = new URLSearchParams(window.location.search);
 el.siblings().removeClass('active');
 el.addClass('active');
+if ($(this).hasClass("versions")) {
+$('.current-version').html( $(this).text() + ' ' );
+if (!$(this).text().includes(defaultVersion)) {
+if (!window.location.search.includes("version")) {
+history.pushState(null, null, '/install/index.html' + 
window.location.search.concat( '=' + $(this).text() ));
+} else {
+history.pushState(null, null, '/install/index.html' + 
window.location.search.replace( urlParams.get('version'), $(this).text() ));
+}
+} else if (window.location.search.includes("version")) {
+  history.pushState(null, null, '/install/index.html' + 
window.location.search.replace( 'version', 'prev' ));
+  }
+}
+else if ($(this).hasClass("Devices")) {
+history.pushState(null, null, '/install/index.html' + 
window.location.search.replace( urlParams.get('device'), $(this).text() ));
+}
+else if ($(this).hasClass("languages")) {
+history.pushState(null, null, '/install/index.html' + 
window.location.search.replace( urlParams.get('language'), $(this).text() ));
+}
+else if ($(this).hasClass("processors")) {
+history.pushState(null, null, '/install/index.html' + 
window.location.search.replace( urlParams.get('processor'), $(this).text() ));
+}
 showContent();
+//window.location.search = 

[incubator-mxnet] branch master updated: [MXNET-504] Add version select + queryString capabilities + C++ instructions to install page (#11128)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 794074d  [MXNET-504] Add version select + queryString capabilities + 
C++ instructions to install page (#11128)
794074d is described below

commit 794074dafded7a935112f074a693dabf74601af5
Author: kpmurali <37911926+kpmur...@users.noreply.github.com>
AuthorDate: Thu Jun 7 14:41:12 2018 -0700

[MXNET-504] Add version select + queryString capabilities + C++ 
instructions to install page (#11128)

* Intial modifications to the MXNet install page versions select

* Adding the acutal install instructions for Linux-Python-CPU-Virtualenv 
path

* Adding the acutal install instructions for Linux-Python-CPU-Pip path

* Adding the acutal install instructions for Linux-Python-GPU-Pip & 
Virtualenvs paths

* Adding master version to the selector in the Install page. Added the 
instructions for the MacOS-Python-CPU-Pip path

* Adding master version to the selector in the Install page. Added the 
instructions for the MacOS-Python-CPU-Virtual env path

* Adding the acutal install instructions for Windows-Python-CPU-Pip

* Adding the acutal install instructions for Windows-Python-GPU-Pip

* Consolidating install instructions for 0.12.1 and 0.12.0

* Making the install page version select available under all scenarios

* In install page, removing the version specifier in pip install for 1.2.0

* Adding the dropd-down version selector to the install page

* Fixed the version display on the install page versions selector

* Minor fixes to the new versions dropdown in the install page

* Add logic for queryString on the install page

* Adding the logic for version selector for processors

* Fix to show C++ instructions in the install page

* Enhancing the queryString capability in version select to use the default 
version unless an older version is explicitly selected

* Add C++ installation instruction options
---
 docs/_static/js/options.js |  65 -
 docs/install/index.md  | 701 +++--
 2 files changed, 742 insertions(+), 24 deletions(-)

diff --git a/docs/_static/js/options.js b/docs/_static/js/options.js
index 8fe74ee..d2d44b0 100644
--- a/docs/_static/js/options.js
+++ b/docs/_static/js/options.js
@@ -1,10 +1,47 @@
-//$('.burger-link').on('click', function(e) { e.stopPropagation() });
-//$('.burger-link').on('touchstart', function(e) { e.stopPropagation() });
+var versionSelect   = defaultVersion = 'v1.2.0';
+var deviceSelect= 'Linux';
+var languageSelect  = 'Python';
+var processorSelect = 'CPU';
+var environSelect   = 'Pip';
 
 $(document).ready(function () {
 function label(lbl) {
 return lbl.replace(/[ .]/g, '-').toLowerCase();
 }
+
+function setSelects(){
+let urlParams = new URLSearchParams(window.location.search);
+if (urlParams.get('version'))
+versionSelect = urlParams.get('version');
+$('li a:contains(' + versionSelect + 
')').parent().siblings().removeClass('active');
+$('li a:contains(' + versionSelect + ')').parent().addClass('active');
+$('.current-version').html( versionSelect + ' ' );
+if (urlParams.get('device'))
+deviceSelect = urlParams.get('device');
+$('button:contains(' + deviceSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + deviceSelect + ')').addClass('active');
+if (urlParams.get('language'))
+languageSelect = urlParams.get('language');
+$('button:contains(' + languageSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + languageSelect + ')').addClass('active');
+if (urlParams.get('processor'))
+processorSelect = urlParams.get('processor');
+$('button:contains(' + processorSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + processorSelect + ')').addClass('active');
+if (urlParams.get('environ'))
+environSelect = urlParams.get('environ');
+$('button:contains(' + environSelect + 
')').siblings().removeClass('active');
+$('button:contains(' + environSelect + ')').addClass('active');
+showContent();
+if (window.location.href.includes("/install/index.html")) {
+if (versionSelect.includes(defaultVersion)) {
+history.pushState(null, null, '/install/index.html?device=' + 
deviceSelect + '=' + languageSelect + '=' + processorSelect);
+} else {
+history.pushState(null, null, '/install/index.html?version=' + 
versionSelect + '=' + deviceSelect + '=' + languageSelect + 
'=' + processorSelect);
+   

[GitHub] szha commented on issue #11128: [MXNET-504] Add version select + queryString capabilities + C++ instructions to install page

[GitBox]

szha commented on issue #11128: [MXNET-504] Add version select + queryString 
capabilities + C++ instructions to install page
URL: https://github.com/apache/incubator-mxnet/pull/11128#issuecomment-395574407
 
 
   
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-11128/5/index.html.
 Checked the installation page and the version drop-down looks good to me.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #11128: [MXNET-504] Add version select + queryString capabilities + C++ instructions to install page

[GitBox]

szha commented on issue #11128: [MXNET-504] Add version select + queryString 
capabilities + C++ instructions to install page
URL: https://github.com/apache/incubator-mxnet/pull/11128#issuecomment-395573819
 
 
   
![image](https://user-images.githubusercontent.com/2626883/41127661-7ed562e4-6a60-11e8-8c1c-b1284485fb8f.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #11128: [MXNET-504] Add version select + queryString capabilities + C++ instructions to install page

[GitBox]

szha commented on issue #11128: [MXNET-504] Add version select + queryString 
capabilities + C++ instructions to install page
URL: https://github.com/apache/incubator-mxnet/pull/11128#issuecomment-395573819
 
 
   
![image](https://user-images.githubusercontent.com/2626883/41127661-7ed562e4-6a60-11e8-8c1c-b1284485fb8f.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu closed pull request #11169: Fix ci/build.py using remote cache locally

[GitBox]

marcoabreu closed pull request #11169: Fix ci/build.py using remote cache 
locally
URL: https://github.com/apache/incubator-mxnet/pull/11169
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/ci/build.py b/ci/build.py
index e52fa794bc9..4fe76eecf31 100755
--- a/ci/build.py
+++ b/ci/build.py
@@ -221,63 +221,69 @@ def script_name() -> str:
 help="go in a shell inside the container",
 action='store_true')
 
-parser.add_argument("--docker-registry",
-help="Dockerhub registry name to retrieve cache from",
+parser.add_argument("-d", "--docker-registry",
+help="Dockerhub registry name to retrieve cache from. 
Default is 'mxnetci'",
 default='mxnetci',
 type=str)
 
+parser.add_argument("-c", "--cache", action="store_true",
+help="Enable docker registry cache")
+
 parser.add_argument("command",
 help="command to run in the container",
 nargs='*', action='append', type=str)
 
 args = parser.parse_args()
-docker_registry = args.docker_registry
+def use_cache():
+return args.cache or 'JOB_NAME' in os.environ # we are in Jenkins
+
 command = list(chain(*args.command))
 docker_binary = get_docker_binary(args.nvidiadocker)
 shared_memory_size = args.shared_memory_size
 
-print("into container: {}".format(args.into_container))
 if args.list:
 list_platforms()
 elif args.platform:
 platform = args.platform
-tag = get_docker_tag(platform=platform, registry=docker_registry)
-load_docker_cache(tag=tag, docker_registry=args.docker_registry)
-build_docker(platform, docker_binary, registry=docker_registry)
+tag = get_docker_tag(platform=platform, registry=args.docker_registry)
+if use_cache():
+load_docker_cache(tag=tag, docker_registry=args.docker_registry)
+build_docker(platform, docker_binary, registry=args.docker_registry)
 if args.build_only:
 logging.warning("Container was just built. Exiting due to 
build-only.")
 return 0
 
 if command:
 container_run(platform=platform, docker_binary=docker_binary, 
shared_memory_size=shared_memory_size,
-  command=command, docker_registry=docker_registry)
+  command=command, 
docker_registry=args.docker_registry)
 elif args.print_docker_run:
 print(container_run(platform=platform, 
docker_binary=docker_binary, shared_memory_size=shared_memory_size,
-command=[], dry_run=True, 
docker_registry=docker_registry))
+command=[], dry_run=True, 
docker_registry=args.docker_registry))
 elif args.into_container:
 container_run(platform=platform, docker_binary=docker_binary, 
shared_memory_size=shared_memory_size,
-  command=[], dry_run=False, into_container=True, 
docker_registry=docker_registry)
+  command=[], dry_run=False, into_container=True, 
docker_registry=args.docker_registry)
 else:
 cmd = ["/work/mxnet/ci/docker/runtime_functions.sh", 
"build_{}".format(platform)]
 logging.info("No command specified, trying default build: %s", ' 
'.join(cmd))
 container_run(platform=platform, docker_binary=docker_binary, 
shared_memory_size=shared_memory_size,
-  command=cmd, docker_registry=docker_registry)
+  command=cmd, docker_registry=args.docker_registry)
 
 elif args.all:
 platforms = get_platforms()
 logging.info("Building for all architectures: {}".format(platforms))
 logging.info("Artifacts will be produced in the build/ directory.")
 for platform in platforms:
-tag = get_docker_tag(platform=platform, registry=docker_registry)
-load_docker_cache(tag=tag, docker_registry=args.docker_registry)
-build_docker(platform, docker_binary)
+tag = get_docker_tag(platform=platform, 
registry=args.docker_registry)
+if use_cache():
+load_docker_cache(tag=tag, 
docker_registry=args.docker_registry)
+build_docker(platform, docker_binary, args.docker_registry)
 if args.build_only:
 continue
 build_platform = "build_{}".format(platform)
 cmd = ["/work/mxnet/ci/docker/runtime_functions.sh", 
build_platform]
 shutil.rmtree(buildir(), ignore_errors=True)
 container_run(platform=platform, 

[GitHub] nswamy commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)

[GitBox]

nswamy commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)
URL: https://github.com/apache/incubator-mxnet/pull/11142#issuecomment-395567826
 
 
   Please wait for https://github.com/apache/incubator-mxnet/pull/11147 to be 
merged.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)

[GitBox]

nswamy commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)
URL: https://github.com/apache/incubator-mxnet/pull/11142#issuecomment-395565921
 
 
   @eric-haibin-lin I am handling the version bump for Scala on a separate PR 
https://github.com/apache/incubator-mxnet/pull/11147. Let me know if you want 
me to revert that. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)

[GitBox]

nswamy commented on issue #11142: [MXNET-408] inplace ReLU activation (#10847)
URL: https://github.com/apache/incubator-mxnet/pull/11142#issuecomment-395565921
 
 
   @eric-haibin-lin I am handling the version bump for Scala on a separate PR 
https://github.com/apache/incubator-mxnet/pull/11147


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy closed issue #11093: Some wrong with mxnet on spark: params.jars = jars.split(", |:")

[GitBox]

nswamy closed issue #11093: Some wrong with mxnet on spark: params.jars = 
jars.split(",|:")
URL: https://github.com/apache/incubator-mxnet/issues/11093
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on issue #11093: Some wrong with mxnet on spark: params.jars = jars.split(", |:")

[GitBox]

nswamy commented on issue #11093: Some wrong with mxnet on spark: params.jars = 
jars.split(",|:")
URL: 
https://github.com/apache/incubator-mxnet/issues/11093#issuecomment-395565290
 
 
   Not an issue at the moment and We will add a Feature Request to support jars 
from S3 and HDFS.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

nswamy commented on a change in pull request #11147: changes made to publish to 
maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193887873
 
 

 ##
 File path: scala-package/dev/compile-mxnet-backend.sh
 ##
 @@ -0,0 +1,107 @@
+#!/bin/bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# (Yizhi) This is mainly inspired by the script in apache/spark.
+# I did some modificaiton to get it with our project.
+#
+
+set -e
+echo "Compiling MXNet Backend, Hang tight!."
+
+if [[ ($# -ne 3) || ( $1 == "--help") ||  $1 == "-h" ]]; then
+  echo "Usage: [-h|--help]   
" 1>&2
+  exit 1
+fi
+PLATFORM=$1
+MXNETDIR=$2
+
+
+# below routine shamelessly copied from
+# 
https://github.com/apache/incubator-mxnet/blob/master/setup-utils/install-mxnet-osx-python.sh
+
+chkret() {
+   cmd=$*
+   echo "$cmd"
+   $cmd
+   ret=$?
+   if [[ ${ret} != 0 ]]; then
+   echo " "
+   echo "ERROR: Return value non-zero for: $cmd"
+   echo " "
+   exit 1
+   fi
+} # chkret()
+
+UNAME=`uname -s`
+chkret pushd $MXNETDIR
+chkret git submodule update --init --recursive
+
+# don't want to overwrite an existing config file
+cp make/config.mk ./config.mk
+
+if [[ $PLATFORM == "osx-x86_64-cpu" ]];
+then
+echo "Building MXNet Backend on MAC OS"
+echo "ADD_CFLAGS += -I/usr/local/opt/opencv@2/include" >> ./config.mk
 
 Review comment:
   Yes, lets fix this in the next iteration during release. I don't know at the 
moment what will work.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on a change in pull request #11147: changes made to publish to maven

[GitBox]

nswamy commented on a change in pull request #11147: changes made to publish to 
maven
URL: https://github.com/apache/incubator-mxnet/pull/11147#discussion_r193887720
 
 

 ##
 File path: scala-package/pom.xml
 ##
 @@ -30,12 +30,13 @@
 scm:git:g...@github.com:apache/incubator-mxnet.git
 
scm:git:g...@github.com:apache/incubator-mxnet.git
 https://github.com/apache/incubator-mxnet
-mxnet-parent_2.11-1.2.0
+HEAD
   
 
   
 2.11.8
 2.11
+
 
 Review comment:
   this is passed from the Makefile. ie., to say no default value


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhreshold closed pull request #11033: Add frame-resize option to ssd demo to scale camera input

[GitBox]

zhreshold closed pull request #11033: Add frame-resize option to ssd demo to 
scale camera input
URL: https://github.com/apache/incubator-mxnet/pull/11033
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/example/ssd/README.md b/example/ssd/README.md
index 55387c5fd2d..cc034689c7b 100644
--- a/example/ssd/README.md
+++ b/example/ssd/README.md
@@ -17,7 +17,8 @@ remarkable traits of MXNet.
 Due to the permission issue, this example is maintained in this 
[repository](https://github.com/zhreshold/mxnet-ssd) separately. You can use 
the link regarding specific per example 
[issues](https://github.com/zhreshold/mxnet-ssd/issues).
 
 ### What's new
-* Added live camera capture and detection display (run with --camera flag)
+* Added live camera capture and detection display (run with --camera flag). 
Example:
+`./demo.py --camera --cpu --frame-resize 0.5`
 * Added multiple trained models.
 * Added a much simpler way to compose network from mainstream classification 
networks (resnet, inception...) and [Guide](symbol/README.md).
 * Update to the latest version according to caffe version, with 5% mAP 
increase.
diff --git a/example/ssd/dataset/cv2Iterator.py 
b/example/ssd/dataset/cv2Iterator.py
index 469faeac828..0af8c3272fa 100644
--- a/example/ssd/dataset/cv2Iterator.py
+++ b/example/ssd/dataset/cv2Iterator.py
@@ -26,10 +26,19 @@ class CameraIterator():
 """
 def __init__(self, capture=cv2.VideoCapture(0), frame_resize=None):
 self._capture = capture
-self._frame_resize = frame_resize
+self._frame_resize = None
 if frame_resize:
-assert isinstance(frame_resize, tuple) and (len(tuple) == 2), 
"frame_resize should be a tuple of (x,y)"
-self._frame_shape = (1, 3, frame_resize[0], frame_resize[1])
+if isinstance(frame_resize, (tuple, list)) and (len(frame_resize) 
== 2):
+self._frame_resize = tuple(map(int, frame_resize))
+self._frame_shape = (1, 3, self._frame_resize[0], 
self._frame_resize[1])
+elif isinstance(frame_resize, float):
+width = 
int(self._capture.get(cv2.CAP_PROP_FRAME_WIDTH)*frame_resize)
+height = 
int(self._capture.get(cv2.CAP_PROP_FRAME_HEIGHT)*frame_resize)
+self._frame_shape = (1, 3, width, height)
+self._frame_resize = (width, height)
+else:
+assert False, "frame_resize should be a tuple of (x,y) pixels "
+"or a float setting the scaling factor"
 else:
 self._frame_shape = (1, 3,
 int(self._capture.get(cv2.CAP_PROP_FRAME_WIDTH)),
diff --git a/example/ssd/demo.py b/example/ssd/demo.py
index 4ae8b350742..e8194ab8ead 100755
--- a/example/ssd/demo.py
+++ b/example/ssd/demo.py
@@ -109,6 +109,8 @@ def parse_args():
 help='string of comma separated names, or text 
filename')
 parser.add_argument('--camera', action='store_true',
 help="use camera for image capturing")
+parser.add_argument('--frame-resize', type=str, default=None,
+help="resize camera frame to x,y pixels or a float 
scaling factor")
 args = parser.parse_args()
 return args
 
@@ -127,6 +129,15 @@ def parse_class_names(class_names):
 raise RuntimeError("No valid class_name provided...")
 return class_names
 
+def parse_frame_resize(x):
+if not x:
+return x
+x = list(map(float, x.strip().split(',')))
+assert len(x) >= 1 and len(x) <= 2, "frame_resize should be a float 
scaling factor or a tuple of w,h pixels"
+if len(x) == 1:
+x = x[0]
+return x
+
 def parse_data_shape(data_shape_str):
 """Parse string to tuple or int"""
 ds = data_shape_str.strip().split(',')
@@ -160,7 +171,7 @@ def network_path(prefix, network, data_shape):
 def run_camera(args,ctx):
 assert args.batch_size == 1, "only batch size of 1 is supported"
 logging.info("Detection threshold is {}".format(args.thresh))
-iter = CameraIterator()
+iter = CameraIterator(frame_resize=parse_frame_resize(args.frame_resize))
 class_names = parse_class_names(args.class_names)
 mean_pixels = (args.mean_r, args.mean_g, args.mean_b)
 data_shape = int(args.data_shape)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services