[GitHub] haojin2 commented on a change in pull request #10714: [MXNET-364] broadcast_add/sub between CSR and 1D dense vector on CPU

[GitBox]

haojin2 commented on a change in pull request #10714: [MXNET-364] 
broadcast_add/sub between CSR and 1D dense vector on CPU
URL: https://github.com/apache/incubator-mxnet/pull/10714#discussion_r186936926
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_broadcast_op_basic.cc
 ##
 @@ -89,6 +91,8 @@ Example::
 
 
 Review comment:
   Updated


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #10857: Fixed divison by zero bug in DistanceWeightedSampling in gluon example

[GitBox]

szha commented on a change in pull request #10857: Fixed divison by zero bug in 
DistanceWeightedSampling in gluon example
URL: https://github.com/apache/incubator-mxnet/pull/10857#discussion_r186934386
 
 

 ##
 File path: example/gluon/embedding_learning/model.py
 ##
 @@ -110,7 +110,10 @@ def hybrid_forward(self, F, x):
 mask[i:i+k, i:i+k] = 0
 
 weights = weights * F.array(mask) * (distance < 
self.nonzero_loss_cutoff)
-weights = weights / F.sum(weights, axis=1, keepdims=True)
+weight_norm = F.sum(weights, axis=1, keepdims=True)
+# we need to change all zero elements to 1 to avoid division by zero
+weight_norm = F.where(weight_norm == 0, F.ones_like(weight_norm), 
weight_norm)
 
 Review comment:
   Feels like a hack. ping @chaoyuaw


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] HaoLiuHust opened a new issue #10860: train acc sometimes get nan

[GitBox]

HaoLiuHust opened a new issue #10860: train acc sometimes get nan
URL: https://github.com/apache/incubator-mxnet/issues/10860
 
 
   according to https://github.com/apache/incubator-mxnet/issues/4253, this 
issue should have been fixed, but I still encounter it, my version is 
1.2.0(build on 5.1)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wkcn commented on issue #9823: RCNN example fails for using latest mxnet

[GitBox]

wkcn commented on issue #9823: RCNN example fails for using latest mxnet
URL: 
https://github.com/apache/incubator-mxnet/issues/9823#issuecomment-387618573
 
 
   It seems the reason is that CUDNN call fails.
   
   When the compile options includes USE_CUDNN=1, the softmax activation 
operator uses CUDNNSoftmax.
   
   And the convolution operator prints the log: 
   src/operator/nn/convolution.cu:140: This convolution is not supported by 
cudnn, MXNET convolution is applied.
   
   Softmax Operator dosen't use CUDNN, so it doesn't cause any error.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] spidyDev commented on a change in pull request #10726: Fix for cpp examples in mxnet/cpp-package.

[GitBox]

spidyDev commented on a change in pull request #10726: Fix for cpp examples in 
mxnet/cpp-package.
URL: https://github.com/apache/incubator-mxnet/pull/10726#discussion_r186928289
 
 

 ##
 File path: cpp-package/example/alexnet.cpp
 ##
 @@ -306,19 +310,20 @@ int main(int argc, char const *argv[]) {
 LG << "ITER: " << iter << " Val LogLoss: " << logloss_val.Get();
 
 /*save the parameters*/
-stringstream ss;
-ss << iter;
-string iter_str;
-ss >> iter_str;
-string save_path_param = "./model/alex_param_" + iter_str;
-auto save_args = args_map;
-/*we do not want to save the data and label*/
-save_args.erase(save_args.find("data"));
-save_args.erase(save_args.find("label"));
-/*the alexnet does not get any aux array, so we do not need to save
- * aux_map*/
-LG << "ITER: " << iter << " Saving to..." << save_path_param;
-NDArray::Save(save_path_param, save_args);
+// Need to be fixed
 
 Review comment:
   I have the fix in the change.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #9496: Draw labels name

[GitBox]

szha commented on issue #9496: Draw labels name
URL: https://github.com/apache/incubator-mxnet/pull/9496#issuecomment-387616542
 
 
   @lvaleriu ping. Could you do a rebase?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #10461: allow user to define unknown token symbol

[GitBox]

szha commented on a change in pull request #10461: allow user to define unknown 
token symbol
URL: https://github.com/apache/incubator-mxnet/pull/10461#discussion_r186928145
 
 

 ##
 File path: python/mxnet/rnn/io.py
 ##
 @@ -58,6 +62,8 @@ def encode_sentences(sentences, vocab=None, 
invalid_label=-1, invalid_key='\n',
 if vocab is None:
 vocab = {invalid_key: invalid_label}
 new_vocab = True
+elif unknown_token:
+new_vocab = True
 
 Review comment:
   Ping


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin opened a new pull request #10859: Add Util Function for Memory Plan Inspection

[GitBox]

eric-haibin-lin opened a new pull request #10859: Add Util Function for Memory 
Plan Inspection
URL: https://github.com/apache/incubator-mxnet/pull/10859
 
 
   ## Description ##
   Add some utility function which is useful for inspecting the memory planning 
for the computation graph. This is for developers only. 
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed pull request #10746: [MXNET-368] Fix mxnet::Context hash for 32 bit architectures, where size_t is 32 …

[GitBox]

szha closed pull request #10746: [MXNET-368] Fix mxnet::Context hash for 32 bit 
architectures, where size_t is 32 …
URL: https://github.com/apache/incubator-mxnet/pull/10746
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/include/mxnet/base.h b/include/mxnet/base.h
index 38fd7edd1f3..7cabfe5027b 100644
--- a/include/mxnet/base.h
+++ b/include/mxnet/base.h
@@ -365,7 +365,10 @@ constexpr size_t kMKLDNNAlign = 64;
 namespace std {
 template<> struct hash {
   size_t operator()(const mxnet::Context& ctx) const {
-return (static_cast(ctx.dev_type) << 32) | ctx.dev_id;
+size_t res = 0;
+res = dmlc::HashCombine(res, static_cast(ctx.dev_type));
+res = dmlc::HashCombine(res, static_cast(ctx.dev_id));
+return res;
   }
 };
 }
diff --git a/tests/cpp/misc/base.cc b/tests/cpp/misc/base.cc
new file mode 100644
index 000..b560f02a2a9
--- /dev/null
+++ b/tests/cpp/misc/base.cc
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include 
+#include 
+
+using namespace mxnet;
+using namespace std;
+
+/*
+ * Test that different Context have different hash values
+ */
+TEST(ContextHashTest, ContextHashUnique) {
+set hashes;
+size_t collision_count = 0;
+size_t total = 0;
+for (size_t dev_type = 0; dev_type < 32; ++dev_type) {
+for (size_t dev_id = 0; dev_id < 64; ++dev_id) {
+auto ctx = 
Context::Create(static_cast(dev_type), dev_id);
+size_t res = std::hash()(ctx);
+auto insert_res = hashes.insert(res);
+if (!insert_res.second)
+++collision_count;
+++total;
+}
+}
+double collision = collision_count / static_cast(total);
+cout << "mxnet::Context std::hash collision ratio: " << collision << endl;
+EXPECT_LE(collision, 0.04);
+}
diff --git a/tests/cpp/test_main.cc b/tests/cpp/test_main.cc
index fc46dff1d9b..592a0361efd 100644
--- a/tests/cpp/test_main.cc
+++ b/tests/cpp/test_main.cc
@@ -53,19 +53,21 @@ bool csv = false;
 #if MXNET_USE_CUDA
 
 static bool checkForWorkingCuda() {
-  int count = 0;
-  if (cudaSuccess == cudaGetDeviceCount()) {
-if (count == 0) return -1;
-for (int device = 0; device < count; ++device) {
+  int device_count = 0;
+  bool workingCuda = false;
+  if (cudaSuccess == cudaGetDeviceCount(_count)) {
+for (int device = 0; device < device_count; ++device) {
   cudaDeviceProp prop;
   if (cudaSuccess == cudaGetDeviceProperties(, device)) {
-std::printf("%d.%d ", prop.major, prop.minor);
-return true;
+std::cout << "Found CUDA Device #: " << device << " properties: " << 
prop.major
+  << "." << prop.minor << std::endl;
+workingCuda = true;
   }
 }
   }
-  std::cerr << "Warning: Could not find working CUDA driver" << std::endl;
-  return false;
+  if (!workingCuda)
+std::cerr << "Warning: Could not find working CUDA device" << std::endl;
+  return workingCuda;
 }
 #else
 static bool checkForWorkingCuda() {


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: Fix mxnet::Context hash for 32 bit architectures, where size_t is 32 bit. (#10746)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 3ae01e4  Fix mxnet::Context hash for 32 bit architectures, where 
size_t is 32 bit. (#10746)
3ae01e4 is described below

commit 3ae01e466e3363e50d39a4a9d8cd5777c0399958
Author: Pedro Larroy <928489+lar...@users.noreply.github.com>
AuthorDate: Wed May 9 06:17:30 2018 +0200

Fix mxnet::Context hash for 32 bit architectures, where size_t is 32 bit. 
(#10746)

Improve output of checkForWorkingCuda which was outputting just a number in 
stdout
---
 include/mxnet/base.h   |  5 -
 tests/cpp/misc/base.cc | 46 ++
 tests/cpp/test_main.cc | 18 ++
 3 files changed, 60 insertions(+), 9 deletions(-)

diff --git a/include/mxnet/base.h b/include/mxnet/base.h
index 38fd7ed..7cabfe5 100644
--- a/include/mxnet/base.h
+++ b/include/mxnet/base.h
@@ -365,7 +365,10 @@ constexpr size_t kMKLDNNAlign = 64;
 namespace std {
 template<> struct hash {
   size_t operator()(const mxnet::Context& ctx) const {
-return (static_cast(ctx.dev_type) << 32) | ctx.dev_id;
+size_t res = 0;
+res = dmlc::HashCombine(res, static_cast(ctx.dev_type));
+res = dmlc::HashCombine(res, static_cast(ctx.dev_id));
+return res;
   }
 };
 }
diff --git a/tests/cpp/misc/base.cc b/tests/cpp/misc/base.cc
new file mode 100644
index 000..b560f02
--- /dev/null
+++ b/tests/cpp/misc/base.cc
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include 
+#include 
+
+using namespace mxnet;
+using namespace std;
+
+/*
+ * Test that different Context have different hash values
+ */
+TEST(ContextHashTest, ContextHashUnique) {
+set hashes;
+size_t collision_count = 0;
+size_t total = 0;
+for (size_t dev_type = 0; dev_type < 32; ++dev_type) {
+for (size_t dev_id = 0; dev_id < 64; ++dev_id) {
+auto ctx = 
Context::Create(static_cast(dev_type), dev_id);
+size_t res = std::hash()(ctx);
+auto insert_res = hashes.insert(res);
+if (!insert_res.second)
+++collision_count;
+++total;
+}
+}
+double collision = collision_count / static_cast(total);
+cout << "mxnet::Context std::hash collision ratio: " << collision << endl;
+EXPECT_LE(collision, 0.04);
+}
diff --git a/tests/cpp/test_main.cc b/tests/cpp/test_main.cc
index fc46dff..592a036 100644
--- a/tests/cpp/test_main.cc
+++ b/tests/cpp/test_main.cc
@@ -53,19 +53,21 @@ bool csv = false;
 #if MXNET_USE_CUDA
 
 static bool checkForWorkingCuda() {
-  int count = 0;
-  if (cudaSuccess == cudaGetDeviceCount()) {
-if (count == 0) return -1;
-for (int device = 0; device < count; ++device) {
+  int device_count = 0;
+  bool workingCuda = false;
+  if (cudaSuccess == cudaGetDeviceCount(_count)) {
+for (int device = 0; device < device_count; ++device) {
   cudaDeviceProp prop;
   if (cudaSuccess == cudaGetDeviceProperties(, device)) {
-std::printf("%d.%d ", prop.major, prop.minor);
-return true;
+std::cout << "Found CUDA Device #: " << device << " properties: " << 
prop.major
+  << "." << prop.minor << std::endl;
+workingCuda = true;
   }
 }
   }
-  std::cerr << "Warning: Could not find working CUDA driver" << std::endl;
-  return false;
+  if (!workingCuda)
+std::cerr << "Warning: Could not find working CUDA device" << std::endl;
+  return workingCuda;
 }
 #else
 static bool checkForWorkingCuda() {

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[incubator-mxnet] branch master updated: fix thread contention caused by openmp (#10820)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 0d5560d  fix thread contention caused by openmp (#10820)
0d5560d is described below

commit 0d5560df88629e790f63463c63018ed61bee4030
Author: Joshua Z. Zhang 
AuthorDate: Tue May 8 21:11:53 2018 -0700

fix thread contention caused by openmp (#10820)

* fix thread contention caused by openmp

* fix namespace

* fix relative include
---
 python/mxnet/gluon/data/dataloader.py   | 19 +--
 src/engine/threaded_engine_perdevice.cc | 16 
 src/initialize.cc   | 19 +++
 3 files changed, 36 insertions(+), 18 deletions(-)

diff --git a/python/mxnet/gluon/data/dataloader.py 
b/python/mxnet/gluon/data/dataloader.py
index 7f09e28..7ef18bd 100644
--- a/python/mxnet/gluon/data/dataloader.py
+++ b/python/mxnet/gluon/data/dataloader.py
@@ -83,6 +83,21 @@ class Queue(multiprocessing.queues.Queue):
 self._recv = self._reader.recv
 
 
+class SimpleQueue(multiprocessing.queues.SimpleQueue):
+"""Wrapper for multiprocessing SimpleQueue that dumps NDArray with shared 
memory.
+   SimpleQueue don't use threading internally.
+"""
+def __init__(self, *args, **kwargs):
+if sys.version_info[0] <= 2:
+super(SimpleQueue, self).__init__(*args, **kwargs)
+else:
+super(SimpleQueue, self).__init__(*args, 
ctx=multiprocessing.get_context(),
+  **kwargs)
+self._reader = ConnectionWrapper(self._reader)
+self._writer = ConnectionWrapper(self._writer)
+self._send = self._writer.send
+self._recv = self._reader.recv
+
 def default_batchify_fn(data):
 """Collate data into batch."""
 if isinstance(data[0], nd.NDArray):
@@ -128,7 +143,7 @@ class _MultiWorkerIter(object):
 self._batchify_fn = batchify_fn
 self._batch_sampler = batch_sampler
 self._key_queue = Queue()
-self._data_queue = Queue(2*self._num_workers)
+self._data_queue = SimpleQueue()
 self._data_buffer = {}
 self._rcvd_idx = 0
 self._sent_idx = 0
@@ -170,10 +185,10 @@ class _MultiWorkerIter(object):
 raise StopIteration
 
 while True:
+self._push_next()
 if self._rcvd_idx in self._data_buffer:
 batch = self._data_buffer.pop(self._rcvd_idx)
 self._rcvd_idx += 1
-self._push_next()
 return batch
 idx, batch = self._data_queue.get()
 self._data_buffer[idx] = batch
diff --git a/src/engine/threaded_engine_perdevice.cc 
b/src/engine/threaded_engine_perdevice.cc
index a822788..2f77380 100644
--- a/src/engine/threaded_engine_perdevice.cc
+++ b/src/engine/threaded_engine_perdevice.cc
@@ -53,22 +53,6 @@ class ThreadedEnginePerDevice : public ThreadedEngine {
 
   ThreadedEnginePerDevice() noexcept(false) {
 this->Start();
-#ifndef _WIN32
-pthread_atfork(
-  []() {
-Engine::Get()->Stop();
-  },
-  []() {
-Engine::Get()->Start();
-  },
-  []() {
-// Make children single threaded since they are typically workers
-dmlc::SetEnv("MXNET_CPU_WORKER_NTHREADS", 1);
-dmlc::SetEnv("OMP_NUM_THREADS", 1);
-OpenMP::Get()->set_enabled(false);
-Engine::Get()->Start();
-  });
-#endif
   }
   ~ThreadedEnginePerDevice() noexcept(false) {
 this->StopNoWait();
diff --git a/src/initialize.cc b/src/initialize.cc
index 69d408d..1fd9262 100644
--- a/src/initialize.cc
+++ b/src/initialize.cc
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include "./engine/openmp.h"
 
 namespace mxnet {
 #if MXNET_USE_SIGNAL_HANDLER && DMLC_LOG_STACK_TRACE
@@ -42,6 +43,24 @@ class LibraryInitializer {
 #if MXNET_USE_SIGNAL_HANDLER && DMLC_LOG_STACK_TRACE
 signal(SIGSEGV, SegfaultLogger);
 #endif
+
+// disable openmp for multithreaded workers
+#ifndef _WIN32
+pthread_atfork(
+  []() {
+Engine::Get()->Stop();
+  },
+  []() {
+Engine::Get()->Start();
+  },
+  []() {
+// Make children single threaded since they are typically workers
+dmlc::SetEnv("MXNET_CPU_WORKER_NTHREADS", 1);
+dmlc::SetEnv("OMP_NUM_THREADS", 1);
+engine::OpenMP::Get()->set_enabled(false);
+Engine::Get()->Start();
+  });
+#endif
   }
 
   static LibraryInitializer* Get();

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] szha closed pull request #10820: fix thread contention caused by openmp

[GitBox]

szha closed pull request #10820: fix thread contention caused by openmp
URL: https://github.com/apache/incubator-mxnet/pull/10820
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/python/mxnet/gluon/data/dataloader.py 
b/python/mxnet/gluon/data/dataloader.py
index 7f09e286742..7ef18bd53d3 100644
--- a/python/mxnet/gluon/data/dataloader.py
+++ b/python/mxnet/gluon/data/dataloader.py
@@ -83,6 +83,21 @@ def __init__(self, *args, **kwargs):
 self._recv = self._reader.recv
 
 
+class SimpleQueue(multiprocessing.queues.SimpleQueue):
+"""Wrapper for multiprocessing SimpleQueue that dumps NDArray with shared 
memory.
+   SimpleQueue don't use threading internally.
+"""
+def __init__(self, *args, **kwargs):
+if sys.version_info[0] <= 2:
+super(SimpleQueue, self).__init__(*args, **kwargs)
+else:
+super(SimpleQueue, self).__init__(*args, 
ctx=multiprocessing.get_context(),
+  **kwargs)
+self._reader = ConnectionWrapper(self._reader)
+self._writer = ConnectionWrapper(self._writer)
+self._send = self._writer.send
+self._recv = self._reader.recv
+
 def default_batchify_fn(data):
 """Collate data into batch."""
 if isinstance(data[0], nd.NDArray):
@@ -128,7 +143,7 @@ def __init__(self, num_workers, dataset, batchify_fn, 
batch_sampler):
 self._batchify_fn = batchify_fn
 self._batch_sampler = batch_sampler
 self._key_queue = Queue()
-self._data_queue = Queue(2*self._num_workers)
+self._data_queue = SimpleQueue()
 self._data_buffer = {}
 self._rcvd_idx = 0
 self._sent_idx = 0
@@ -170,10 +185,10 @@ def __next__(self):
 raise StopIteration
 
 while True:
+self._push_next()
 if self._rcvd_idx in self._data_buffer:
 batch = self._data_buffer.pop(self._rcvd_idx)
 self._rcvd_idx += 1
-self._push_next()
 return batch
 idx, batch = self._data_queue.get()
 self._data_buffer[idx] = batch
diff --git a/src/engine/threaded_engine_perdevice.cc 
b/src/engine/threaded_engine_perdevice.cc
index a8227886f84..2f77380baf8 100644
--- a/src/engine/threaded_engine_perdevice.cc
+++ b/src/engine/threaded_engine_perdevice.cc
@@ -53,22 +53,6 @@ class ThreadedEnginePerDevice : public ThreadedEngine {
 
   ThreadedEnginePerDevice() noexcept(false) {
 this->Start();
-#ifndef _WIN32
-pthread_atfork(
-  []() {
-Engine::Get()->Stop();
-  },
-  []() {
-Engine::Get()->Start();
-  },
-  []() {
-// Make children single threaded since they are typically workers
-dmlc::SetEnv("MXNET_CPU_WORKER_NTHREADS", 1);
-dmlc::SetEnv("OMP_NUM_THREADS", 1);
-OpenMP::Get()->set_enabled(false);
-Engine::Get()->Start();
-  });
-#endif
   }
   ~ThreadedEnginePerDevice() noexcept(false) {
 this->StopNoWait();
diff --git a/src/initialize.cc b/src/initialize.cc
index 69d408d7d87..1fd92628e9b 100644
--- a/src/initialize.cc
+++ b/src/initialize.cc
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include "./engine/openmp.h"
 
 namespace mxnet {
 #if MXNET_USE_SIGNAL_HANDLER && DMLC_LOG_STACK_TRACE
@@ -42,6 +43,24 @@ class LibraryInitializer {
 #if MXNET_USE_SIGNAL_HANDLER && DMLC_LOG_STACK_TRACE
 signal(SIGSEGV, SegfaultLogger);
 #endif
+
+// disable openmp for multithreaded workers
+#ifndef _WIN32
+pthread_atfork(
+  []() {
+Engine::Get()->Stop();
+  },
+  []() {
+Engine::Get()->Start();
+  },
+  []() {
+// Make children single threaded since they are typically workers
+dmlc::SetEnv("MXNET_CPU_WORKER_NTHREADS", 1);
+dmlc::SetEnv("OMP_NUM_THREADS", 1);
+engine::OpenMP::Get()->set_enabled(false);
+Engine::Get()->Start();
+  });
+#endif
   }
 
   static LibraryInitializer* Get();


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #10726: Fix for cpp examples in mxnet/cpp-package.

[GitBox]

szha commented on a change in pull request #10726: Fix for cpp examples in 
mxnet/cpp-package.
URL: https://github.com/apache/incubator-mxnet/pull/10726#discussion_r186926678
 
 

 ##
 File path: cpp-package/example/alexnet.cpp
 ##
 @@ -306,19 +310,20 @@ int main(int argc, char const *argv[]) {
 LG << "ITER: " << iter << " Val LogLoss: " << logloss_val.Get();
 
 /*save the parameters*/
-stringstream ss;
-ss << iter;
-string iter_str;
-ss >> iter_str;
-string save_path_param = "./model/alex_param_" + iter_str;
-auto save_args = args_map;
-/*we do not want to save the data and label*/
-save_args.erase(save_args.find("data"));
-save_args.erase(save_args.find("label"));
-/*the alexnet does not get any aux array, so we do not need to save
- * aux_map*/
-LG << "ITER: " << iter << " Saving to..." << save_path_param;
-NDArray::Save(save_path_param, save_args);
+// Need to be fixed
 
 Review comment:
   what error? are you still trying to fix it?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: consolidated contribute and community info (#10855)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 25bede9  consolidated contribute and community info (#10855)
25bede9 is described below

commit 25bede94d7f1ae3f7fcb328e6c06f3ef3896e714
Author: Aaron Markham 
AuthorDate: Tue May 8 20:28:05 2018 -0700

consolidated contribute and community info (#10855)
---
 docs/_static/mxnet-theme/navbar.html   |   5 +-
 docs/build_version_doc/artifacts/.htaccess |   1 +
 docs/community/contribute.md   | 345 +
 docs/community/index.md|  33 ---
 4 files changed, 151 insertions(+), 233 deletions(-)

diff --git a/docs/_static/mxnet-theme/navbar.html 
b/docs/_static/mxnet-theme/navbar.html
index 218454a..8ea2f9f 100644
--- a/docs/_static/mxnet-theme/navbar.html
+++ b/docs/_static/mxnet-theme/navbar.html
@@ -14,7 +14,7 @@
 http://gluon.mxnet.io;>Tutorials
   
 
- 
+
 
   API 
   
@@ -43,8 +43,7 @@
   Community 
   
 http://discuss.mxnet.io;>Forum
-https://github.com/apache/incubator-mxnet;>Github
-Community
+https://github.com/apache/incubator-mxnet;>Github
 Contribute
 Powered By
   
diff --git a/docs/build_version_doc/artifacts/.htaccess 
b/docs/build_version_doc/artifacts/.htaccess
index 76d0893..5467448 100644
--- a/docs/build_version_doc/artifacts/.htaccess
+++ b/docs/build_version_doc/artifacts/.htaccess
@@ -3,3 +3,4 @@ RewriteRule ^get_started/why_mxnet.html$ /faq/why_mxnet.html 
[R=301,L]
 RewriteRule ^get_started.*$ /install/ [R=301,L]
 RewriteRule ^how_to.*$ /faq/ [R=301,L]
 RewriteRule ^api/python/symbol.html$ /api/python/symbol/symbol.html [R=301,L]
+RewriteRule ^community/index.html$ /community/contribute.html [R=301,L]
diff --git a/docs/community/contribute.md b/docs/community/contribute.md
index e3d10f5..a2d2f64 100644
--- a/docs/community/contribute.md
+++ b/docs/community/contribute.md
@@ -1,202 +1,153 @@
-# Contribute to MXNet
+# Contributing MXNet
 
-MXNet has been developed and is used by a group of active community members.
-Please contribute to improve the project.
-After your patch has been merged, remember to add your name to 
[CONTRIBUTORS.md](https://github.com/apache/incubator-mxnet/blob/master/CONTRIBUTORS.md).
+Apache MXNet (incubating) is a community led, open source deep learning 
project. We welcome new members and look forward to your contributions. Here 
you will find how to get started and links to detailed information on MXNet 
best practices and processes.
 
-## Code Contribution
 
-Before you start coding…
+## Getting Started
 
-… please make sure there is a JIRA issue that corresponds to your 
contribution. This is a general rule that the MXNet community follows for all 
code contributions, including bug fixes, improvements, or new features, with an 
exception for trivial hot fixes. If you would like to fix a bug that you found 
or if you would like to add a new feature or improvement to MXNet, please 
follow the [File a bug report or Propose an improvement or a new 
feature](http://mxnet.io/community/index.html) gui [...]
-
-If the description of a JIRA issue indicates that its resolution will touch 
sensible parts of the code base, be sufficiently complex, or add significant 
amounts of new code, the MXNet community might request a design document (most 
contributions should not require a design document). The purpose of this 
document is to ensure that the overall approach to address the issue is 
sensible and agreed upon by the community. JIRA issues that require a design 
document are tagged with the requires- [...]
-
-- Overview of the general approach
-- List of API changes (changed interfaces, new and deprecated configuration 
parameters, changed behavior, …)
-- Main components and classes to be touched
-- Known limitations of the proposed approach
-
-A design document can be added by anybody, including the reporter of the issue 
or the person working on it.
-
-Contributions for JIRA issues that require a design document will not be added 
to MXNet’s code base before a design document has been accepted by the 
community with lazy consensus. Please check if a design document is required 
before starting to code.
-
-
-### Core Library
-
-- Follow the [Google C++ Style 
Guide](https://google.github.io/styleguide/cppguide.html) for C++ code.
-- Use doxygen to document all of the interface code.
-- Use [RAII](http://en.cppreference.com/w/cpp/language/raii) to manage 
resources, including smart
- pointers like shared_ptr and unique_ptr as well as allocating in constructors 
and deallocating in
- destructors. Avoid explicit calls to new and delete when possible. Use 
make_shared and 

[GitHub] szha closed pull request #10855: consolidated contribute and community info

[GitBox]

szha closed pull request #10855: consolidated contribute and community info
URL: https://github.com/apache/incubator-mxnet/pull/10855
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/_static/mxnet-theme/navbar.html 
b/docs/_static/mxnet-theme/navbar.html
index 218454aea77..8ea2f9f2161 100644
--- a/docs/_static/mxnet-theme/navbar.html
+++ b/docs/_static/mxnet-theme/navbar.html
@@ -14,7 +14,7 @@ 
 http://gluon.mxnet.io;>Tutorials
   
 
- 
+
 
   API 
   
@@ -43,8 +43,7 @@ 
   Community 
   
 http://discuss.mxnet.io;>Forum
-https://github.com/apache/incubator-mxnet;>Github
-Community
+https://github.com/apache/incubator-mxnet;>Github
 Contribute
 Powered By
   
diff --git a/docs/build_version_doc/artifacts/.htaccess 
b/docs/build_version_doc/artifacts/.htaccess
index 76d0893a528..5467448f521 100644
--- a/docs/build_version_doc/artifacts/.htaccess
+++ b/docs/build_version_doc/artifacts/.htaccess
@@ -3,3 +3,4 @@ RewriteRule ^get_started/why_mxnet.html$ /faq/why_mxnet.html 
[R=301,L]
 RewriteRule ^get_started.*$ /install/ [R=301,L]
 RewriteRule ^how_to.*$ /faq/ [R=301,L]
 RewriteRule ^api/python/symbol.html$ /api/python/symbol/symbol.html [R=301,L]
+RewriteRule ^community/index.html$ /community/contribute.html [R=301,L]
diff --git a/docs/community/contribute.md b/docs/community/contribute.md
index e3d10f527dc..a2d2f64616f 100644
--- a/docs/community/contribute.md
+++ b/docs/community/contribute.md
@@ -1,202 +1,153 @@
-# Contribute to MXNet
+# Contributing MXNet
 
-MXNet has been developed and is used by a group of active community members.
-Please contribute to improve the project.
-After your patch has been merged, remember to add your name to 
[CONTRIBUTORS.md](https://github.com/apache/incubator-mxnet/blob/master/CONTRIBUTORS.md).
+Apache MXNet (incubating) is a community led, open source deep learning 
project. We welcome new members and look forward to your contributions. Here 
you will find how to get started and links to detailed information on MXNet 
best practices and processes.
 
-## Code Contribution
 
-Before you start coding…
+## Getting Started
 
-… please make sure there is a JIRA issue that corresponds to your 
contribution. This is a general rule that the MXNet community follows for all 
code contributions, including bug fixes, improvements, or new features, with an 
exception for trivial hot fixes. If you would like to fix a bug that you found 
or if you would like to add a new feature or improvement to MXNet, please 
follow the [File a bug report or Propose an improvement or a new 
feature](http://mxnet.io/community/index.html) guidelines to open an issue in 
[MXNet’s JIRA](http://issues.apache.org/jira/browse/MXNet) before starting with 
the implementation.
-
-If the description of a JIRA issue indicates that its resolution will touch 
sensible parts of the code base, be sufficiently complex, or add significant 
amounts of new code, the MXNet community might request a design document (most 
contributions should not require a design document). The purpose of this 
document is to ensure that the overall approach to address the issue is 
sensible and agreed upon by the community. JIRA issues that require a design 
document are tagged with the requires-design-doc label. The label can be 
attached by any community member who feels that a design document is necessary. 
A good description helps to decide whether a JIRA issue requires a design 
document or not. The design document must be added or attached to or link from 
the JIRA issue and cover the following aspects:
-
-- Overview of the general approach
-- List of API changes (changed interfaces, new and deprecated configuration 
parameters, changed behavior, …)
-- Main components and classes to be touched
-- Known limitations of the proposed approach
-
-A design document can be added by anybody, including the reporter of the issue 
or the person working on it.
-
-Contributions for JIRA issues that require a design document will not be added 
to MXNet’s code base before a design document has been accepted by the 
community with lazy consensus. Please check if a design document is required 
before starting to code.
-
-
-### Core Library
-
-- Follow the [Google C++ Style 
Guide](https://google.github.io/styleguide/cppguide.html) for C++ code.
-- Use doxygen to document all of the interface code.
-- Use [RAII](http://en.cppreference.com/w/cpp/language/raii) to manage 
resources, including smart
- pointers like shared_ptr and unique_ptr as well as allocating in constructors 
and deallocating in
- destructors. Avoid explicit calls to new and delete 

[GitHub] aaronmarkham opened a new issue #10858: make docs on Mac - scala docs failure

[GitBox]

aaronmarkham opened a new issue #10858: make docs on Mac - scala docs failure
URL: https://github.com/apache/incubator-mxnet/issues/10858
 
 
   ## Description
   Making docs on a Mac is broken if you use the latest version of scala, 
maven, and sbt.
   
   ## Minimum reproducible example
   On a Mac, run `make docs`.
   Unrelated, but this actually fails unless you do `make` first.
   
   Then you'll see you need some dependencies like in the [setup_docs_ubuntu 
script](https://github.com/apache/incubator-mxnet/blob/master/docs/build_version_doc/setup_docs_ubuntu.sh)
 and then follow up with these to make sure you're getting the system installs 
for Mac (you must have Sphinx 1.5.6 just like CI):
   ```
   pip install sphinx==1.5.6
   brew install maven
   brew upgrade maven
   brew install sbt
   brew install scala
   make
   make docs
   ```
   That's pretty much what I had to do to get the docs to get even close to 
building on the Mac.
   But, then it will fail when it tries to move some scala docs files that 
don't exist.
   
   ## What have you tried to solve it?
   
   I sort of solved it, but it isn't something I think I can commit because it 
might break what's happening in CI on Ubuntu.
   
   1. Remove `index` and `package.html` from [mxdoc.py line 
89](https://github.com/apache/incubator-mxnet/blob/master/docs/mxdoc.py#L89)
   2. Run `make docs` again and it's fine. Even better than fine, because the 
docs seem to be much more modern and usable than the current docs.
   
   ## Comments
   
   If you get stuck building docs locally with a Mac, do what I did in step 1 
above. 
   I'd like to see how to upgrade the Ubuntu scaladocs in CI to what I'm seeing 
on the Mac. CI docker setup uses no version info when installing the scala 
deps, so it's not obvious at the moment. Acquiring a different version of scala 
deps would need to be thoroughly tested since it could break any number of 
other things. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 115be5f  Bump the publish timestamp.
115be5f is described below

commit 115be5f3598996695d617ff741dfd8f92e97c87a
Author: mxnet-ci 
AuthorDate: Wed May 9 01:56:27 2018 +

Bump the publish timestamp.
---
 date.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/date.txt b/date.txt
new file mode 100644
index 000..32dc6c5
--- /dev/null
+++ b/date.txt
@@ -0,0 +1 @@
+Wed May  9 01:56:27 UTC 2018

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] eric-haibin-lin commented on issue #10855: consolidated contribute and community info

[GitBox]

eric-haibin-lin commented on issue #10855: consolidated contribute and 
community info
URL: https://github.com/apache/incubator-mxnet/pull/10855#issuecomment-387591761
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ifeherva opened a new pull request #10857: Fixed divison by zero bug in DistanceWeightedSampling in gluon example

[GitBox]

ifeherva opened a new pull request #10857: Fixed divison by zero bug in 
DistanceWeightedSampling in gluon example
URL: https://github.com/apache/incubator-mxnet/pull/10857
 
 
   ## Description ##
   Sample selection for training is based on a vector of computed probabilities 
that come from distance weights. In the current implementation these weights 
can become zero (especially at random initialization) when distances to all 
other neighbors are zero.
   Zero weights lead to nan bugs in the model, this commit is supposed to fix 
it by changing those zero weights not being divided by zero any longer.
   
   This example project has no unit test. :(
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yajiedesign commented on issue #10854: "WindowsError: [Error 126]" and "ImportError: cannot import name libinfo"

[GitBox]

yajiedesign commented on issue #10854: "WindowsError: [Error 126]" and 
"ImportError: cannot import name libinfo"
URL: 
https://github.com/apache/incubator-mxnet/issues/10854#issuecomment-387590802
 
 
   you can download https://github.com/lucasg/Dependencies and use it find what 
dll is missing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhreshold closed pull request #10823: [MXNET-403]fix gradient number bug and document for box_iou operator

[GitBox]

zhreshold closed pull request #10823: [MXNET-403]fix gradient number bug and 
document for box_iou operator
URL: https://github.com/apache/incubator-mxnet/pull/10823
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/operator/contrib/bounding_box.cc 
b/src/operator/contrib/bounding_box.cc
index 288fe449734..53052ad5a25 100644
--- a/src/operator/contrib/bounding_box.cc
+++ b/src/operator/contrib/bounding_box.cc
@@ -117,7 +117,7 @@ NNVM_REGISTER_OP(_contrib_box_iou)
   Example::
 
 x = [[0.5, 0.5, 1.0, 1.0], [0.0, 0.0, 0.5, 0.5]]
-y = [0.25, 0.25, 0.75, 0.75]
+y = [[0.25, 0.25, 0.75, 0.75]]
 box_iou(x, y, format='corner') = [[0.1428], [0.1428]]
 
 )doc" ADD_FILELINE)
@@ -137,8 +137,8 @@ NNVM_REGISTER_OP(_contrib_box_iou)
 .add_arguments(BoxOverlapParam::__FIELDS__());
 
 NNVM_REGISTER_OP(_backward_contrib_box_iou)
-.set_num_inputs(2)
-.set_num_outputs(1)
+.set_num_inputs(1)
+.set_num_outputs(2)
 .set_attr_parser(ParamParser)
 .set_attr("TIsBackward", true)
 .set_attr("FCompute", BoxOverlapBackward)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: fix gradient number and document for box_iou operator (#10823)

[zhreshold]

This is an automated email from the ASF dual-hosted git repository.

zhreshold pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 33c58dc  fix gradient number and document for box_iou operator (#10823)
33c58dc is described below

commit 33c58dc95e7fefbe89c52076261935496a79a02b
Author: JackieWu 
AuthorDate: Wed May 9 08:54:58 2018 +0800

fix gradient number and document for box_iou operator (#10823)
---
 src/operator/contrib/bounding_box.cc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/operator/contrib/bounding_box.cc 
b/src/operator/contrib/bounding_box.cc
index 288fe44..53052ad 100644
--- a/src/operator/contrib/bounding_box.cc
+++ b/src/operator/contrib/bounding_box.cc
@@ -117,7 +117,7 @@ NNVM_REGISTER_OP(_contrib_box_iou)
   Example::
 
 x = [[0.5, 0.5, 1.0, 1.0], [0.0, 0.0, 0.5, 0.5]]
-y = [0.25, 0.25, 0.75, 0.75]
+y = [[0.25, 0.25, 0.75, 0.75]]
 box_iou(x, y, format='corner') = [[0.1428], [0.1428]]
 
 )doc" ADD_FILELINE)
@@ -137,8 +137,8 @@ NNVM_REGISTER_OP(_contrib_box_iou)
 .add_arguments(BoxOverlapParam::__FIELDS__());
 
 NNVM_REGISTER_OP(_backward_contrib_box_iou)
-.set_num_inputs(2)
-.set_num_outputs(1)
+.set_num_inputs(1)
+.set_num_outputs(2)
 .set_attr_parser(ParamParser)
 .set_attr("TIsBackward", true)
 .set_attr("FCompute", BoxOverlapBackward)

-- 
To stop receiving notification emails like this one, please contact
zhresh...@apache.org.

[GitHub] zhreshold commented on issue #10854: "WindowsError: [Error 126]" and "ImportError: cannot import name libinfo"

[GitBox]

zhreshold commented on issue #10854: "WindowsError: [Error 126]" and 
"ImportError: cannot import name libinfo"
URL: 
https://github.com/apache/incubator-mxnet/issues/10854#issuecomment-387586469
 
 
   WindowsError: [Error 126] indicates that there's `SOME` missing dlls, either 
not in PATH or missing. Unfortunately, windows won't tell you which one is 
missing. So it's kind of bad experience to debug what's going wrong.
   
   Can you make sure your PATH include cuda dlls and vc++ runtime dlls?
   
   ```
   echo $PATH$
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anirudh2290 opened a new issue #10856: test_multi_worker_forked_data_loader with DEBUG=1

[GitBox]

anirudh2290 opened a new issue #10856: test_multi_worker_forked_data_loader 
with DEBUG=1
URL: https://github.com/apache/incubator-mxnet/issues/10856
 
 
   Reproduction steps:
   ```
   [INFO] Setting module np/mx/python random seeds, use 
MXNET_MODULE_SEED=1107463865 to reproduce.
   Test should successfully run its course of multi-process/forked data loader 
without errors ... Assertion failure at kmp_runtime.cpp(6481): __kmp_team_pool 
== __null.
   OMP: Error #13: Assertion failure at kmp_runtime.cpp(6481).
   OMP: Hint: Please submit a bug report with this message, compile and run 
commands used, and machine configuration info including native compiler and 
operating system versions. Faster response will be obtained by including all 
program sources. For information on submitting this issue, please see 
https://bugs.llvm.org/.
   Assertion failure at kmp_runtime.cpp(6481): __kmp_team_pool == __null.
   OMP: Error #13: Assertion failure at kmp_runtime.cpp(6481).
   OMP: Hint: Please submit a bug report with this message, compile and run 
commands used, and machine configuration info including native compiler and 
operating system versions. Faster response will be obtained by including all 
program sources. For information on submitting this issue, please see 
https://bugs.llvm.org/.
   ^C[INFO] Setting test np/mx/python random seeds, use 
MXNET_TEST_SEED=39684479 to reproduce.
   
   --
   Ran 1 test in 2.106s
   
   OK
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] hetong007 commented on issue #10320: RMSProp doesn't work for MobileNetV2 on ImageNet

[GitBox]

hetong007 commented on issue #10320: RMSProp doesn't work for MobileNetV2 on 
ImageNet
URL: 
https://github.com/apache/incubator-mxnet/issues/10320#issuecomment-387583011
 
 
   @jonmorton thanks, will test! It's just not mentioned in Google's paper.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jonmorton commented on issue #10320: RMSProp doesn't work for MobileNetV2 on ImageNet

[GitBox]

jonmorton commented on issue #10320: RMSProp doesn't work for MobileNetV2 on 
ImageNet
URL: 
https://github.com/apache/incubator-mxnet/issues/10320#issuecomment-387582346
 
 
   You have to use an eps value of 1.0 for RMSprop to work. Doesn't make any 
sense, but it's necessary. You can see find this value in Google's code if you 
look closely enough.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham commented on issue #10855: consolidated contribute and community info

[GitBox]

aaronmarkham commented on issue #10855: consolidated contribute and community 
info
URL: https://github.com/apache/incubator-mxnet/pull/10855#issuecomment-387580032
 
 
   @eric-haibin-lin Can you please review?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] CodingCat commented on a change in pull request #10462: [MXNET-62] add test against spark integration

[GitBox]

CodingCat commented on a change in pull request #10462: [MXNET-62] add test 
against spark integration
URL: https://github.com/apache/incubator-mxnet/pull/10462#discussion_r186901639
 
 

 ##
 File path: 
scala-package/spark/src/test/scala/org/apache/mxnet/spark/MXNetGeneralSuite.scala
 ##
 @@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.spark
+
+import org.apache.spark.SparkContext
+import org.apache.spark.mllib.linalg.Vectors
+import org.apache.spark.mllib.regression.LabeledPoint
+import org.apache.spark.rdd.RDD
+
+class MXNetGeneralSuite extends SharedSparkContext {
+
+  private def parseRawData(sc: SparkContext, path: String): RDD[LabeledPoint] 
= {
+val raw = sc.textFile(path)
+raw.map { s =>
+  val parts = s.split(' ')
+  val label = java.lang.Double.parseDouble(parts(0))
+  val features = 
Vectors.dense(parts(1).trim().split(',').map(java.lang.Double.parseDouble))
+  LabeledPoint(label, features)
+}
+  }
+
+  test("run spark with MLP") {
+val trainData = parseRawData(sc,
+  "/Users/nanzhu/code/mxnet/scala-package/spark/train.txt")
 
 Review comment:
   Hi, @szha, would you please help to uncompress and reupload, I zipped it 
just because github does not allow a large file...I am not sure if jenkins env 
has unzip command line equipped, etc.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] aaronmarkham opened a new pull request #10855: consolidated contribute and community info

[GitBox]

aaronmarkham opened a new pull request #10855: consolidated contribute and 
community info
URL: https://github.com/apache/incubator-mxnet/pull/10855
 
 
   ## Description ##
   
   Contribution info was spread out on the contribute, community, and 
Confluence wiki pages. I have migrated and updated much of the contribution 
information.
   
   ### Changes ###
   - [x] Moved content from index.html to contribute.html
   - [x] Moved detailed content from contribute.html to the confluence wiki 
/development pages
   - [x] Clarified contribution resources and added a getting started guide
   - [x] Deleted index.html and added a redirect to contribute.html
   - [x] Removed the Community > Community navigation
   
   ### Preview
   http://34.235.144.251/community/contribute.html
   
   If you're looking for content that moved to the wiki:
   * https://cwiki.apache.org/confluence/display/MXNET/MXNet+Development+Guide
   * https://cwiki.apache.org/confluence/display/MXNET/Development+Process
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on a change in pull request #10462: [MXNET-62] add test against spark integration

[GitBox]

szha commented on a change in pull request #10462: [MXNET-62] add test against 
spark integration
URL: https://github.com/apache/incubator-mxnet/pull/10462#discussion_r186900742
 
 

 ##
 File path: 
scala-package/spark/src/test/scala/org/apache/mxnet/spark/MXNetGeneralSuite.scala
 ##
 @@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.spark
+
+import org.apache.spark.SparkContext
+import org.apache.spark.mllib.linalg.Vectors
+import org.apache.spark.mllib.regression.LabeledPoint
+import org.apache.spark.rdd.RDD
+
+class MXNetGeneralSuite extends SharedSparkContext {
+
+  private def parseRawData(sc: SparkContext, path: String): RDD[LabeledPoint] 
= {
+val raw = sc.textFile(path)
+raw.map { s =>
+  val parts = s.split(' ')
+  val label = java.lang.Double.parseDouble(parts(0))
+  val features = 
Vectors.dense(parts(1).trim().split(',').map(java.lang.Double.parseDouble))
+  LabeledPoint(label, features)
+}
+  }
+
+  test("run spark with MLP") {
+val trainData = parseRawData(sc,
+  "/Users/nanzhu/code/mxnet/scala-package/spark/train.txt")
 
 Review comment:
   
http://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/mxnet-spark-test/train.txt.zip


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] whria78 opened a new issue #10854: "WindowsError: [Error 126]" and "ImportError: cannot import name libinfo"

[GitBox]

whria78 opened a new issue #10854: "WindowsError: [Error 126]" and 
"ImportError: cannot import name libinfo"
URL: https://github.com/apache/incubator-mxnet/issues/10854
 
 
   Note: Providing complete information in the most concise form is the best 
way to get help. This issue template serves as the checklist for essential 
information to most of the technical issues and bug reports. For non-technical 
issues and feature requests, feel free to present the information in what you 
believe is the best form.
   
   For Q & A and discussion, please start a discussion thread at 
https://discuss.mxnet.io 
   
   ## Description
   (Brief description of the problem in no more than 2 sentences.)
   
   The first error is "WindowsError: [Error 126]"
   The second error is "ImportError: cannot import name libinfo"
   
   ## Environment info (Required)
   
   Windows 10 64bit
   Python 2.7.14
   pip install mxnet-cu91, mxnet-cu90, mxnet-cu80  ;  whereas mxnet CPU version 
works well.
   
   I tested with 2 computers.
   
   ## Error Message:
   (Paste the complete error message, including stack trace.)
   
   ```
   C:\Users\whria>pip install mxnet-cu91
   Collecting mxnet-cu91
 Using cached 
https://files.pythonhosted.org/packages/56/20/92638b338feb87f54f9d943a4bf5c2075e65b590915e5c0de1387a96ab76/mxnet_cu91-1.1.0.post0-py2.py3-none-win_amd64.whl
   Requirement already satisfied: requests in c:\anaconda2\lib\site-packages 
(from mxnet-cu91)
   Requirement already satisfied: numpy in c:\anaconda2\lib\site-packages (from 
mxnet-cu91)
   Requirement already satisfied: graphviz in c:\anaconda2\lib\site-packages 
(from mxnet-cu91)
   Requirement already satisfied: chardet<3.1.0,>=3.0.2 in 
c:\anaconda2\lib\site-packages (from requests->mxnet-cu91)
   Requirement already satisfied: idna<2.7,>=2.5 in 
c:\anaconda2\lib\site-packages (from requests->mxnet-cu91)
   Requirement already satisfied: urllib3<1.23,>=1.21.1 in 
c:\anaconda2\lib\site-packages (from requests->mxnet-cu91)
   Requirement already satisfied: certifi>=2017.4.17 in 
c:\anaconda2\lib\site-packages (from requests->mxnet-cu91)
   Installing collected packages: mxnet-cu91
   Successfully installed mxnet-cu91-1.1.0.post0
   
   C:\Users\whria>python
   Python 2.7.14 | packaged by conda-forge | (default, Mar 30 2018, 18:30:38) 
[MSC v.1500 64 bit (AMD64)] on win32
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import mxnet
   Traceback (most recent call last):
 File "", line 1, in 
 File "C:\Anaconda2\lib\site-packages\mxnet\__init__.py", line 25, in 

   from . import engine
 File "C:\Anaconda2\lib\site-packages\mxnet\engine.py", line 23, in 
   from .base import _LIB, check_call
 File "C:\Anaconda2\lib\site-packages\mxnet\base.py", line 111, in 
   _LIB = _load_lib()
 File "C:\Anaconda2\lib\site-packages\mxnet\base.py", line 103, in _load_lib
   lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
 File "C:\Anaconda2\lib\ctypes\__init__.py", line 366, in __init__
   self._handle = _dlopen(self._name, mode)
   WindowsError: [Error 126]
   >>> import mxnet
   Traceback (most recent call last):
 File "", line 1, in 
 File "C:\Anaconda2\lib\site-packages\mxnet\__init__.py", line 25, in 

   from . import engine
 File "C:\Anaconda2\lib\site-packages\mxnet\engine.py", line 23, in 
   from .base import _LIB, check_call
 File "C:\Anaconda2\lib\site-packages\mxnet\base.py", line 30, in 
   from . import libinfo
   ImportError: cannot import name libinfo
   
   ```
   
   The first error is "WindowsError: [Error 126]"
   The second error is "ImportError: cannot import name libinfo"
   
   ## What have you tried to solve it?
   
   I added windows PATH ; I also tried to add os.environment['path'] in python 
script.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #10580: inference results unstable in mxnet_mkl-1.2.0b20180416

[GitBox]

szha closed issue #10580: inference results unstable in 
mxnet_mkl-1.2.0b20180416 
URL: https://github.com/apache/incubator-mxnet/issues/10580
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha opened a new pull request #10853: update mshadow

[GitBox]

szha opened a new pull request #10853: update mshadow
URL: https://github.com/apache/incubator-mxnet/pull/10853
 
 
   ## Description ##
   update mshadow to fix some build problems.
   
   ## Checklist ##
   ### Essentials ###
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [x] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] update mshadow


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed pull request #10849: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub

[GitBox]

szha closed pull request #10849: [MXNET-409] Temporary fix for ARM64 builds - 
switch to own dockerhub
URL: https://github.com/apache/incubator-mxnet/pull/10849
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/ci/docker/Dockerfile.build.arm64 b/ci/docker/Dockerfile.build.arm64
index eb68a818ba6..ec949600f73 100755
--- a/ci/docker/Dockerfile.build.arm64
+++ b/ci/docker/Dockerfile.build.arm64
@@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
-ENV CC /usr/bin/aarch64-linux-gnu-gcc
-ENV CXX /usr/bin/aarch64-linux-gnu-g++
-ENV FC /usr/bin/aarch64-linux-gnu-gfortran-4.9
+ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
 ENV HOSTCC gcc
+ENV TARGET ARMV8
 
 WORKDIR /work
 
-COPY install/arm64_openblas.sh /work/
-RUN /work/arm64_openblas.sh
+# Build OpenBLAS
+RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
+cd OpenBLAS && \
+make -j$(nproc) && \
+PREFIX=${CROSS_ROOT} make install
 
-ENV LD_LIBRARY_PATH /opt/OpenBLAS/lib
-ENV CPLUS_INCLUDE_PATH /opt/OpenBLAS/include
+COPY runtime_functions.sh /work/
 WORKDIR /work/mxnet
-
-COPY runtime_functions.sh /work/
\ No newline at end of file
diff --git a/ci/docker/Dockerfile.build.jetson 
b/ci/docker/Dockerfile.build.jetson
index 9fa50f4097a..c358edb1fb0 100755
--- a/ci/docker/Dockerfile.build.jetson
+++ b/ci/docker/Dockerfile.build.jetson
@@ -22,7 +22,9 @@
 
 FROM nvidia/cuda:9.0-cudnn7-devel as cudabuilder
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+# FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
 ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
@@ -32,7 +34,6 @@ ENV TARGET ARMV8
 WORKDIR /work
 
 # Build OpenBLAS
-ADD https://api.github.com/repos/xianyi/OpenBLAS/git/refs/tags/v0.2.20 
openblas_version.json
 RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
 cd OpenBLAS && \
 make -j$(nproc) && \


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch v1.2.0 updated: Move from dockcross to own mirror (#10850)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch v1.2.0
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/v1.2.0 by this push:
 new bf78e48  Move from dockcross to own mirror (#10850)
bf78e48 is described below

commit bf78e48ac60bbadeafd691bd3e5dfb63610c9aed
Author: Marco de Abreu 
AuthorDate: Wed May 9 01:16:10 2018 +0200

Move from dockcross to own mirror (#10850)
---
 ci/docker/Dockerfile.build.arm64  | 21 +++--
 ci/docker/Dockerfile.build.jetson |  5 +++--
 2 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/ci/docker/Dockerfile.build.arm64 b/ci/docker/Dockerfile.build.arm64
index eb68a81..ec94960 100755
--- a/ci/docker/Dockerfile.build.arm64
+++ b/ci/docker/Dockerfile.build.arm64
@@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
-ENV CC /usr/bin/aarch64-linux-gnu-gcc
-ENV CXX /usr/bin/aarch64-linux-gnu-g++
-ENV FC /usr/bin/aarch64-linux-gnu-gfortran-4.9
+ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
 ENV HOSTCC gcc
+ENV TARGET ARMV8
 
 WORKDIR /work
 
-COPY install/arm64_openblas.sh /work/
-RUN /work/arm64_openblas.sh
+# Build OpenBLAS
+RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
+cd OpenBLAS && \
+make -j$(nproc) && \
+PREFIX=${CROSS_ROOT} make install
 
-ENV LD_LIBRARY_PATH /opt/OpenBLAS/lib
-ENV CPLUS_INCLUDE_PATH /opt/OpenBLAS/include
+COPY runtime_functions.sh /work/
 WORKDIR /work/mxnet
-
-COPY runtime_functions.sh /work/
\ No newline at end of file
diff --git a/ci/docker/Dockerfile.build.jetson 
b/ci/docker/Dockerfile.build.jetson
index 9fa50f4..c358edb 100755
--- a/ci/docker/Dockerfile.build.jetson
+++ b/ci/docker/Dockerfile.build.jetson
@@ -22,7 +22,9 @@
 
 FROM nvidia/cuda:9.0-cudnn7-devel as cudabuilder
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+# FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
 ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
@@ -32,7 +34,6 @@ ENV TARGET ARMV8
 WORKDIR /work
 
 # Build OpenBLAS
-ADD https://api.github.com/repos/xianyi/OpenBLAS/git/refs/tags/v0.2.20 
openblas_version.json
 RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
 cd OpenBLAS && \
 make -j$(nproc) && \

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[incubator-mxnet] branch master updated: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub (#10849)

[zhasheng]

This is an automated email from the ASF dual-hosted git repository.

zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 4aff7de  [MXNET-409] Temporary fix for ARM64 builds - switch to own 
dockerhub (#10849)
4aff7de is described below

commit 4aff7defae1d88eccd49d1a4d01c799dd4477b8d
Author: Marco de Abreu 
AuthorDate: Wed May 9 01:16:36 2018 +0200

[MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub 
(#10849)

* Move to own dockcross repository

* switch account
---
 ci/docker/Dockerfile.build.arm64  | 21 +++--
 ci/docker/Dockerfile.build.jetson |  5 +++--
 2 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/ci/docker/Dockerfile.build.arm64 b/ci/docker/Dockerfile.build.arm64
index eb68a81..ec94960 100755
--- a/ci/docker/Dockerfile.build.arm64
+++ b/ci/docker/Dockerfile.build.arm64
@@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
-ENV CC /usr/bin/aarch64-linux-gnu-gcc
-ENV CXX /usr/bin/aarch64-linux-gnu-g++
-ENV FC /usr/bin/aarch64-linux-gnu-gfortran-4.9
+ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
 ENV HOSTCC gcc
+ENV TARGET ARMV8
 
 WORKDIR /work
 
-COPY install/arm64_openblas.sh /work/
-RUN /work/arm64_openblas.sh
+# Build OpenBLAS
+RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
+cd OpenBLAS && \
+make -j$(nproc) && \
+PREFIX=${CROSS_ROOT} make install
 
-ENV LD_LIBRARY_PATH /opt/OpenBLAS/lib
-ENV CPLUS_INCLUDE_PATH /opt/OpenBLAS/include
+COPY runtime_functions.sh /work/
 WORKDIR /work/mxnet
-
-COPY runtime_functions.sh /work/
\ No newline at end of file
diff --git a/ci/docker/Dockerfile.build.jetson 
b/ci/docker/Dockerfile.build.jetson
index 9fa50f4..c358edb 100755
--- a/ci/docker/Dockerfile.build.jetson
+++ b/ci/docker/Dockerfile.build.jetson
@@ -22,7 +22,9 @@
 
 FROM nvidia/cuda:9.0-cudnn7-devel as cudabuilder
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+# FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
 ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
@@ -32,7 +34,6 @@ ENV TARGET ARMV8
 WORKDIR /work
 
 # Build OpenBLAS
-ADD https://api.github.com/repos/xianyi/OpenBLAS/git/refs/tags/v0.2.20 
openblas_version.json
 RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
 cd OpenBLAS && \
 make -j$(nproc) && \

-- 
To stop receiving notification emails like this one, please contact
zhash...@apache.org.

[GitHub] szha closed pull request #10850: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub (1.2.0)

[GitBox]

szha closed pull request #10850: [MXNET-409] Temporary fix for ARM64 builds - 
switch to own dockerhub (1.2.0)
URL: https://github.com/apache/incubator-mxnet/pull/10850
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/ci/docker/Dockerfile.build.arm64 b/ci/docker/Dockerfile.build.arm64
index eb68a818ba6..ec949600f73 100755
--- a/ci/docker/Dockerfile.build.arm64
+++ b/ci/docker/Dockerfile.build.arm64
@@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
-ENV CC /usr/bin/aarch64-linux-gnu-gcc
-ENV CXX /usr/bin/aarch64-linux-gnu-g++
-ENV FC /usr/bin/aarch64-linux-gnu-gfortran-4.9
+ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
 ENV HOSTCC gcc
+ENV TARGET ARMV8
 
 WORKDIR /work
 
-COPY install/arm64_openblas.sh /work/
-RUN /work/arm64_openblas.sh
+# Build OpenBLAS
+RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
+cd OpenBLAS && \
+make -j$(nproc) && \
+PREFIX=${CROSS_ROOT} make install
 
-ENV LD_LIBRARY_PATH /opt/OpenBLAS/lib
-ENV CPLUS_INCLUDE_PATH /opt/OpenBLAS/include
+COPY runtime_functions.sh /work/
 WORKDIR /work/mxnet
-
-COPY runtime_functions.sh /work/
\ No newline at end of file
diff --git a/ci/docker/Dockerfile.build.jetson 
b/ci/docker/Dockerfile.build.jetson
index 9fa50f4097a..c358edb1fb0 100755
--- a/ci/docker/Dockerfile.build.jetson
+++ b/ci/docker/Dockerfile.build.jetson
@@ -22,7 +22,9 @@
 
 FROM nvidia/cuda:9.0-cudnn7-devel as cudabuilder
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+# FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 ENV ARCH aarch64
 ENV FC /usr/bin/${CROSS_TRIPLE}-gfortran
@@ -32,7 +34,6 @@ ENV TARGET ARMV8
 WORKDIR /work
 
 # Build OpenBLAS
-ADD https://api.github.com/repos/xianyi/OpenBLAS/git/refs/tags/v0.2.20 
openblas_version.json
 RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
 cd OpenBLAS && \
 make -j$(nproc) && \


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10591: [MXNET-365] handle inplace in mkldnn FallBackCompute

[GitBox]

ashokei commented on issue #10591: [MXNET-365] handle inplace in mkldnn 
FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#issuecomment-387569021
 
 
   @marcoabreu we now have CI running though these test with `-- build` change 
you suggested above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #10382: Does memonger work for gluon to save memory?

[GitBox]

eric-haibin-lin commented on issue #10382: Does memonger work for gluon to save 
memory?
URL: 
https://github.com/apache/incubator-mxnet/issues/10382#issuecomment-387563352
 
 
   We did done some memory optimization recently. You might want to check it 
out:
   https://github.com/apache/incubator-mxnet/pull/10847


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #10847: [MXNET-408] [WIP] inplace ReLU activation

[GitBox]

piiswrong closed pull request #10847: [MXNET-408] [WIP] inplace ReLU activation
URL: https://github.com/apache/incubator-mxnet/pull/10847
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/operator/nn/activation-inl.h b/src/operator/nn/activation-inl.h
index 32a7a5ad617..a9f6dbeda89 100644
--- a/src/operator/nn/activation-inl.h
+++ b/src/operator/nn/activation-inl.h
@@ -83,7 +83,7 @@ struct hash {
 namespace mxnet {
 namespace op {
 
-template
+template
 void ActivationForward(const OpContext , const TBlob _data,
const OpReqType , const TBlob _data) {
   using namespace mshadow;
@@ -91,16 +91,16 @@ void ActivationForward(const OpContext , const TBlob 
_data,
   Stream *s = ctx.get_stream();
   const size_t sz = in_data.shape_.Size();
   if (sz) {
-MXNET_ASSIGN_REQ_SWITCH(req, Req, {
-  mxnet_op::Kernel, xpu>::Launch(
-s, sz,
-out_data.dptr(),
-in_data.dptr());
+MSHADOW_REAL_TYPE_SWITCH(in_data.type_flag_, DType, {
+  MXNET_ASSIGN_REQ_SWITCH(req, Req, {
+mxnet_op::Kernel, xpu>::Launch(
+  s, sz, out_data.dptr(), in_data.dptr());
+  });
 });
   }
 }
 
-template
+template
 void ActivationBackward(const OpContext , const TBlob _grad,
 const TBlob _data, const OpReqType ,
 const TBlob _grad) {
@@ -109,13 +109,12 @@ void ActivationBackward(const OpContext , const TBlob 
_grad,
   Stream *s = ctx.get_stream();
   const size_t sz = out_data.shape_.Size();
   if (sz) {
-MXNET_ASSIGN_REQ_SWITCH(req, Req, {
-  mxnet_op::Kernel, 
xpu>::Launch(
-s, sz,
-in_grad.dptr(),
-out_grad.dptr(),
-out_data.dptr());
+MSHADOW_REAL_TYPE_SWITCH(out_grad.type_flag_, DType, {
+  MXNET_ASSIGN_REQ_SWITCH(req, Req, {
+mxnet_op::Kernel, xpu>::Launch(
+s, sz, in_grad.dptr(), out_grad.dptr(), 
out_data.dptr());
+  });
 });
   }
 }
@@ -123,72 +122,68 @@ void ActivationBackward(const OpContext , const TBlob 
_grad,
 template
 void ActivationComputeImpl(const ActivationParam , const OpContext ,
const TBlob , OpReqType req, const TBlob 
) {
-  MSHADOW_REAL_TYPE_SWITCH(input.type_flag_, DType, {
-switch (param.act_type) {
-  case activation::kReLU:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kSigmoid:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kTanh:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kSoftReLU:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kSoftSign:
-ActivationForward(
-ctx, input, req, output);
-break;
-  default:
-LOG(FATAL) << "unknown activation type";
-}
-  });
+  switch (param.act_type) {
+case activation::kReLU:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+case activation::kSigmoid:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+case activation::kTanh:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+case activation::kSoftReLU:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+case activation::kSoftSign:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+default:
+  LOG(FATAL) << "unknown activation type";
+  }
 }
 
 template
 void ActivationGradComputeImpl(const ActivationParam , const OpContext 
,
const TBlob _grad, const TBlob _data,
OpReqType req, const TBlob ) {
-  MSHADOW_REAL_TYPE_SWITCH(out_grad.type_flag_, DType, {
-switch (param.act_type) {
-  case activation::kReLU:
-ActivationBackward

[incubator-mxnet] branch master updated: [MXNET-408] [WIP] inplace ReLU activation (#10847)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new b2ec05b  [MXNET-408] [WIP] inplace ReLU activation (#10847)
b2ec05b is described below

commit b2ec05b5ec2164c5d47c040d4bebbe4ff6a1cb8f
Author: Haibin Lin 
AuthorDate: Tue May 8 15:10:36 2018 -0700

[MXNET-408] [WIP] inplace ReLU activation (#10847)

* inplace version of activation(relu)

* inplace relu

* add comments

* add commnet

* comments

* fix compilation error

* add check_numerical_grad test
---
 src/operator/nn/activation-inl.h   | 148 -
 src/operator/nn/activation.cc  |  50 ++---
 src/operator/nn/activation.cu  |  39 ---
 src/operator/nn/cudnn/cudnn_activation-inl.h   |   3 +
 src/operator/nn/mkldnn/mkldnn_act.cc   |   2 +
 src/operator/tensor/elemwise_unary_op_basic.cc |   2 +-
 tests/python/gpu/test_operator_gpu.py  |  19 ++--
 tests/python/unittest/test_operator.py |  42 +++
 8 files changed, 187 insertions(+), 118 deletions(-)

diff --git a/src/operator/nn/activation-inl.h b/src/operator/nn/activation-inl.h
index 32a7a5a..a9f6dbe 100644
--- a/src/operator/nn/activation-inl.h
+++ b/src/operator/nn/activation-inl.h
@@ -83,7 +83,7 @@ struct hash {
 namespace mxnet {
 namespace op {
 
-template
+template
 void ActivationForward(const OpContext , const TBlob _data,
const OpReqType , const TBlob _data) {
   using namespace mshadow;
@@ -91,16 +91,16 @@ void ActivationForward(const OpContext , const TBlob 
_data,
   Stream *s = ctx.get_stream();
   const size_t sz = in_data.shape_.Size();
   if (sz) {
-MXNET_ASSIGN_REQ_SWITCH(req, Req, {
-  mxnet_op::Kernel, xpu>::Launch(
-s, sz,
-out_data.dptr(),
-in_data.dptr());
+MSHADOW_REAL_TYPE_SWITCH(in_data.type_flag_, DType, {
+  MXNET_ASSIGN_REQ_SWITCH(req, Req, {
+mxnet_op::Kernel, xpu>::Launch(
+  s, sz, out_data.dptr(), in_data.dptr());
+  });
 });
   }
 }
 
-template
+template
 void ActivationBackward(const OpContext , const TBlob _grad,
 const TBlob _data, const OpReqType ,
 const TBlob _grad) {
@@ -109,13 +109,12 @@ void ActivationBackward(const OpContext , const TBlob 
_grad,
   Stream *s = ctx.get_stream();
   const size_t sz = out_data.shape_.Size();
   if (sz) {
-MXNET_ASSIGN_REQ_SWITCH(req, Req, {
-  mxnet_op::Kernel, 
xpu>::Launch(
-s, sz,
-in_grad.dptr(),
-out_grad.dptr(),
-out_data.dptr());
+MSHADOW_REAL_TYPE_SWITCH(out_grad.type_flag_, DType, {
+  MXNET_ASSIGN_REQ_SWITCH(req, Req, {
+mxnet_op::Kernel, xpu>::Launch(
+s, sz, in_grad.dptr(), out_grad.dptr(), 
out_data.dptr());
+  });
 });
   }
 }
@@ -123,72 +122,68 @@ void ActivationBackward(const OpContext , const TBlob 
_grad,
 template
 void ActivationComputeImpl(const ActivationParam , const OpContext ,
const TBlob , OpReqType req, const TBlob 
) {
-  MSHADOW_REAL_TYPE_SWITCH(input.type_flag_, DType, {
-switch (param.act_type) {
-  case activation::kReLU:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kSigmoid:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kTanh:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kSoftReLU:
-ActivationForward(
-ctx, input, req, output);
-break;
-  case activation::kSoftSign:
-ActivationForward(
-ctx, input, req, output);
-break;
-  default:
-LOG(FATAL) << "unknown activation type";
-}
-  });
+  switch (param.act_type) {
+case activation::kReLU:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+case activation::kSigmoid:
+  ActivationForward(
+  ctx, input, req, output);
+  break;
+case activation::kTanh:
+  ActivationForward

[GitHub] anirudhacharya commented on issue #10789: Reshape of input array when the shape is available only at runtime is not possible

[GitBox]

anirudhacharya commented on issue #10789: Reshape of input array when the shape 
is available only at runtime is not possible 
URL: 
https://github.com/apache/incubator-mxnet/issues/10789#issuecomment-387551054
 
 
   @larroy Netron - https://github.com/lutzroeder/Netron


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on a change in pull request #10591: [MXNET-365] handle inplace in mkldnn FallBackCompute

[GitBox]

ashokei commented on a change in pull request #10591: [MXNET-365] handle 
inplace in mkldnn FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#discussion_r186873923
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -304,10 +304,15 @@ void FallBackCompute(FCompute fn, const nnvm::NodeAttrs 
,
 
   std::vector out_blobs(outputs.size());
   for (size_t i = 0; i < out_blobs.size(); i++) {
-if (req[i] == kWriteTo)
-  const_cast(outputs[i]).InvalidateMKLDNNData();
-CHECK(outputs[i].IsDefaultData());
-out_blobs[i] = outputs[i].data();
+NDArray output = outputs[i];
+// ensure output does not use mkldnn mem.
+// for inplace, we already converted & copied input above.
+if ((req[i] == kWriteTo) || (req[i] == kWriteInplace))
+  const_cast(output).InvalidateMKLDNNData();
 
 Review comment:
   called in sequence, this PR only adds `kWriteInplace` option to existing 
`kWriteTo` handling.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on issue #10838: Flaky test_random.test_sample_multinomial

[GitBox]

haojin2 commented on issue #10838: Flaky test_random.test_sample_multinomial
URL: 
https://github.com/apache/incubator-mxnet/issues/10838#issuecomment-387547071
 
 
   @marcoabreu here's the link to the failed build: 
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10533/34/pipeline,
 will also update the issue description


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #10591: [MXNET-365] handle inplace in mkldnn FallBackCompute

[GitBox]

marcoabreu commented on a change in pull request #10591: [MXNET-365] handle 
inplace in mkldnn FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#discussion_r186872307
 
 

 ##
 File path: src/operator/nn/mkldnn/mkldnn_base.cc
 ##
 @@ -304,10 +304,15 @@ void FallBackCompute(FCompute fn, const nnvm::NodeAttrs 
,
 
   std::vector out_blobs(outputs.size());
   for (size_t i = 0; i < out_blobs.size(); i++) {
-if (req[i] == kWriteTo)
-  const_cast(outputs[i]).InvalidateMKLDNNData();
-CHECK(outputs[i].IsDefaultData());
-out_blobs[i] = outputs[i].data();
+NDArray output = outputs[i];
+// ensure output does not use mkldnn mem.
+// for inplace, we already converted & copied input above.
+if ((req[i] == kWriteTo) || (req[i] == kWriteInplace))
+  const_cast(output).InvalidateMKLDNNData();
 
 Review comment:
   Is this threadsafe? What happens if multiple threads try to read/write or 
they call it in sequence? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10591: [MXNET-365] handle inplace in mkldnn FallBackCompute

[GitBox]

marcoabreu commented on issue #10591: [MXNET-365] handle inplace in mkldnn 
FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#issuecomment-387546527
 
 
   Let me just retrigger CI


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10838: Flaky test_random.test_sample_multinomial

[GitBox]

marcoabreu commented on issue #10838: Flaky test_random.test_sample_multinomial
URL: 
https://github.com/apache/incubator-mxnet/issues/10838#issuecomment-387546302
 
 
   Hi @haojin2, please also add information about the configuration in which 
this is happening - otherwise, it will be hard to reproduce the failure. 
Ideally, just link a CI result.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10591: [MXNET-365] handle inplace in mkldnn FallBackCompute

[GitBox]

ashokei commented on issue #10591: [MXNET-365] handle inplace in mkldnn 
FallBackCompute
URL: https://github.com/apache/incubator-mxnet/pull/10591#issuecomment-387546149
 
 
   @marcoabreu this one is ready for merge too, made changes suggested by you. 
thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: [MXNET-367] update mkldnn to v0.14 and disable building test examples (#10819)

[marcoabreu]

This is an automated email from the ASF dual-hosted git repository.

marcoabreu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 0fbbe19  [MXNET-367] update mkldnn to v0.14 and disable building test 
examples (#10819)
0fbbe19 is described below

commit 0fbbe19f513d4dd46ad26a0cd5ec13c48edf3f64
Author: Ashok Emani 
AuthorDate: Tue May 8 14:18:49 2018 -0700

[MXNET-367] update mkldnn to v0.14 and disable building test examples 
(#10819)

* update mkldnn to v0.14 and disable building test examples

* use git submodule tagging for mkldnn

* update mklml to latest version
---
 ci/docker/install/ubuntu_mklml.sh | 4 ++--
 prepare_mkl.sh| 6 +++---
 prepare_mkldnn.sh | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/ci/docker/install/ubuntu_mklml.sh 
b/ci/docker/install/ubuntu_mklml.sh
index 253cf95..3689aad 100755
--- a/ci/docker/install/ubuntu_mklml.sh
+++ b/ci/docker/install/ubuntu_mklml.sh
@@ -21,5 +21,5 @@
 # the whole docker cache for the image
 
 set -ex
-wget --no-check-certificate -O /tmp/mklml.tgz 
https://github.com/intel/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz
-tar -zxvf /tmp/mklml.tgz && cp -rf mklml_*/* /usr/local/ && rm -rf mklml_*
\ No newline at end of file
+wget --no-check-certificate -O /tmp/mklml.tgz 
https://github.com/intel/mkl-dnn/releases/download/v0.14/mklml_lnx_2018.0.3.20180406.tgz
+tar -zxvf /tmp/mklml.tgz && cp -rf mklml_*/* /usr/local/ && rm -rf mklml_*
diff --git a/prepare_mkl.sh b/prepare_mkl.sh
index 12e5df7..b702b06 100755
--- a/prepare_mkl.sh
+++ b/prepare_mkl.sh
@@ -58,16 +58,16 @@ MXNET_ROOT=`dirname $0`
 USE_MKLML=0
 # NOTE: if you update the following line, please also update the dockerfile at
 # tests/ci_build/Dockerfile.mkl
-VERSION_MATCH=20171227
+VERSION_MATCH=20180406
 PLATFORM=$(uname)
 if [ $PLATFORM == "Darwin" ]; then
 INFIX=mac
 elif [ $PLATFORM == "Linux" ]; then
 INFIX=lnx
 fi
-ARCHIVE_BASENAME=mklml_${INFIX}_2018.0.1.${VERSION_MATCH}.tgz
+ARCHIVE_BASENAME=mklml_${INFIX}_2018.0.3.${VERSION_MATCH}.tgz
 MKL_CONTENT_DIR=`echo $ARCHIVE_BASENAME | rev | cut -d "." -f 2- | rev`
-MKLURL="https://github.com/01org/mkl-dnn/releases/download/v0.12/$ARCHIVE_BASENAME;
+MKLURL="https://github.com/intel/mkl-dnn/releases/download/v0.14/$ARCHIVE_BASENAME;
 # there are diffrent MKL lib to be used for GCC and for ICC
 reg='^[0-9]+$'
 VERSION_LINE=`GetVersionName $MKLROOT`
diff --git a/prepare_mkldnn.sh b/prepare_mkldnn.sh
index 9b11b4a..b8d5c7e 100755
--- a/prepare_mkldnn.sh
+++ b/prepare_mkldnn.sh
@@ -93,7 +93,7 @@ if [ ! -f $MKLDNN_LIBFILE ]; then
 echo "Building MKLDNN ..." >&2
 cd $MXNET_ROOTDIR
g++ --version >&2
-cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR 
-B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" >&2
+cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR 
-B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" -DWITH_TEST=OFF 
-DWITH_EXAMPLE=OFF >&2
 NUM_PROC=1
 if [[ ! -z $(command -v nproc) ]]; then
   NUM_PROC=$(nproc)

-- 
To stop receiving notification emails like this one, please contact
marcoab...@apache.org.

[GitHub] marcoabreu closed pull request #10819: [MXNET-367] update mkldnn to v0.14 and disable building test examples

[GitBox]

marcoabreu closed pull request #10819: [MXNET-367] update mkldnn to v0.14 and 
disable building test examples
URL: https://github.com/apache/incubator-mxnet/pull/10819
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/ci/docker/install/ubuntu_mklml.sh 
b/ci/docker/install/ubuntu_mklml.sh
index 253cf95c6ce..3689aad65cf 100755
--- a/ci/docker/install/ubuntu_mklml.sh
+++ b/ci/docker/install/ubuntu_mklml.sh
@@ -21,5 +21,5 @@
 # the whole docker cache for the image
 
 set -ex
-wget --no-check-certificate -O /tmp/mklml.tgz 
https://github.com/intel/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz
-tar -zxvf /tmp/mklml.tgz && cp -rf mklml_*/* /usr/local/ && rm -rf mklml_*
\ No newline at end of file
+wget --no-check-certificate -O /tmp/mklml.tgz 
https://github.com/intel/mkl-dnn/releases/download/v0.14/mklml_lnx_2018.0.3.20180406.tgz
+tar -zxvf /tmp/mklml.tgz && cp -rf mklml_*/* /usr/local/ && rm -rf mklml_*
diff --git a/prepare_mkl.sh b/prepare_mkl.sh
index 12e5df7ffe1..b702b06e849 100755
--- a/prepare_mkl.sh
+++ b/prepare_mkl.sh
@@ -58,16 +58,16 @@ MXNET_ROOT=`dirname $0`
 USE_MKLML=0
 # NOTE: if you update the following line, please also update the dockerfile at
 # tests/ci_build/Dockerfile.mkl
-VERSION_MATCH=20171227
+VERSION_MATCH=20180406
 PLATFORM=$(uname)
 if [ $PLATFORM == "Darwin" ]; then
 INFIX=mac
 elif [ $PLATFORM == "Linux" ]; then
 INFIX=lnx
 fi
-ARCHIVE_BASENAME=mklml_${INFIX}_2018.0.1.${VERSION_MATCH}.tgz
+ARCHIVE_BASENAME=mklml_${INFIX}_2018.0.3.${VERSION_MATCH}.tgz
 MKL_CONTENT_DIR=`echo $ARCHIVE_BASENAME | rev | cut -d "." -f 2- | rev`
-MKLURL="https://github.com/01org/mkl-dnn/releases/download/v0.12/$ARCHIVE_BASENAME;
+MKLURL="https://github.com/intel/mkl-dnn/releases/download/v0.14/$ARCHIVE_BASENAME;
 # there are diffrent MKL lib to be used for GCC and for ICC
 reg='^[0-9]+$'
 VERSION_LINE=`GetVersionName $MKLROOT`
diff --git a/prepare_mkldnn.sh b/prepare_mkldnn.sh
index 828cfe107ed..d210d64573a 100755
--- a/prepare_mkldnn.sh
+++ b/prepare_mkldnn.sh
@@ -93,7 +93,7 @@ if [ ! -f $MKLDNN_LIBFILE ]; then
 echo "Building MKLDNN ..." >&2
 cd $MXNET_ROOTDIR
g++ --version >&2
-cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR 
-B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" >&2
+cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR 
-B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" -DWITH_TEST=OFF 
-DWITH_EXAMPLE=OFF >&2
 NUM_PROC=1
 if [[ ! -z $(command -v nproc) ]]; then
   NUM_PROC=$(nproc)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10819: [MXNET-367] update mkldnn to v0.14 and disable building test examples

[GitBox]

marcoabreu commented on issue #10819: [MXNET-367] update mkldnn to v0.14 and 
disable building test examples
URL: https://github.com/apache/incubator-mxnet/pull/10819#issuecomment-387545642
 
 
   Sure, thanks for following up on this :)
   
   Last test was 19hours ago - I think we're fine this time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10842: Update expected result in osx python install script

[GitBox]

marcoabreu commented on issue #10842: Update expected result in osx python 
install script
URL: https://github.com/apache/incubator-mxnet/pull/10842#issuecomment-387544814
 
 
   Hi, thanks for fixing this! 
   Could you elaborate where this is being used?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10850: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub (1.2.0)

[GitBox]

marcoabreu commented on issue #10850: [MXNET-409] Temporary fix for ARM64 
builds - switch to own dockerhub (1.2.0)
URL: https://github.com/apache/incubator-mxnet/pull/10850#issuecomment-387543754
 
 
   @piiswrong @eric-haibin-lin @szha could you approve please?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on issue #10849: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub

[GitBox]

marcoabreu commented on issue #10849: [MXNET-409] Temporary fix for ARM64 
builds - switch to own dockerhub
URL: https://github.com/apache/incubator-mxnet/pull/10849#issuecomment-387543703
 
 
   @piiswrong @eric-haibin-lin @szha could you approve please?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #10104: [WIP][MXNET-107] Fused RNN implementation for CPU

[GitBox]

piiswrong commented on issue #10104: [WIP][MXNET-107] Fused RNN implementation 
for CPU
URL: https://github.com/apache/incubator-mxnet/pull/10104#issuecomment-387526162
 
 
   @TaoLv I'll look into this later. I think you can do it similar to RNNCell.
   Let's merge the backend LSTM implementation first. Is there enough test?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed issue #9310: model using contrib.SparseEmbedding returns inconsistent result between runs

[GitBox]

eric-haibin-lin closed issue #9310: model using contrib.SparseEmbedding returns 
inconsistent result between runs
URL: https://github.com/apache/incubator-mxnet/issues/9310
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #9310: model using contrib.SparseEmbedding returns inconsistent result between runs

[GitBox]

eric-haibin-lin commented on issue #9310: model using contrib.SparseEmbedding 
returns inconsistent result between runs
URL: 
https://github.com/apache/incubator-mxnet/issues/9310#issuecomment-387522000
 
 
   Fixed in https://github.com/apache/incubator-mxnet/pull/9882


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling

[GitBox]

haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling 
and Global Lp Pooling
URL: https://github.com/apache/incubator-mxnet/pull/10780#discussion_r186846020
 
 

 ##
 File path: tests/python/gpu/test_operator_gpu.py
 ##
 @@ -745,7 +745,7 @@ def test_pooling_with_type():
 @with_seed()
 def test_pooling_versions():
 def test_pooling_versions_helper(pool_op_list, data, kernel, pool_type, 
pad, stride,
- pooling_convention='valid', 
global_pool=False):
+ pooling_convention='valid', 
global_pool=False, p_value=2):
 
 Review comment:
   The most commonly used case for Lp pooling is p=2, for all other types of 
pooling this value is not used at all, so there's no side effect on other types 
of pooling with setting p_value to be 2.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alrigazzi commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory and then crash down.

[GitBox]

alrigazzi commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory 
and then crash down.
URL: 
https://github.com/apache/incubator-mxnet/issues/10599#issuecomment-387518590
 
 
   I simply ran the script in examples/image_classification/ but I can give you 
the exact command line. 
   Yes, I built on the same system. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] alrigazzi commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory and then crash down.

[GitBox]

alrigazzi commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory 
and then crash down.
URL: 
https://github.com/apache/incubator-mxnet/issues/10599#issuecomment-387518590
 
 
   I simply ran the script in examples/image_classification/ but I can give you 
the exact command line. 
   Yes, I built on the same system. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #9851: Softsign op

[GitBox]

eric-haibin-lin commented on a change in pull request #9851: Softsign op
URL: https://github.com/apache/incubator-mxnet/pull/9851#discussion_r186826380
 
 

 ##
 File path: src/operator/tensor/elemwise_unary_op_basic.cc
 ##
 @@ -106,6 +106,23 @@ The storage type of ``sigmoid`` output is always dense
 
 MXNET_OPERATOR_REGISTER_BINARY_WITH_SPARSE_CPU(_backward_sigmoid,

unary_bwd);
+// softsign
+MXNET_OPERATOR_REGISTER_UNARY(softsign)
+MXNET_ADD_SPARSE_OP_ALIAS(softsign)
 
 Review comment:
   This operator doesn't have a sparse implementation. Sparse op alias should 
not be registered. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #9851: Softsign op

[GitBox]

eric-haibin-lin commented on a change in pull request #9851: Softsign op
URL: https://github.com/apache/incubator-mxnet/pull/9851#discussion_r186827581
 
 

 ##
 File path: tests/python/unittest/test_operator.py
 ##
 @@ -486,6 +486,21 @@ def fsigmoid(a):
 check_symbolic_forward(y, [xa], [ya])
 check_symbolic_backward(y, [xa], [np.ones(shape)], [ya * (1 - ya)])
 
+@with_seed()
+def test_softsign():
 
 Review comment:
   Is there unit test for `Activation(act_type='softsign')`???


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #9851: Softsign op

[GitBox]

eric-haibin-lin commented on a change in pull request #9851: Softsign op
URL: https://github.com/apache/incubator-mxnet/pull/9851#discussion_r186827452
 
 

 ##
 File path: src/operator/nn/activation-inl.h
 ##
 @@ -168,6 +173,10 @@ void ActivationGradComputeImpl(const ActivationParam 
, const OpContext 
 ActivationBackward(
 ctx, out_grad, out_data, req, output);
 break;
+  case activation::kSoftSign:
 
 Review comment:
   Are you sure this is correct? 
   
   `ActivationGradComputeImpl` takes `out_grad` and `out_data` to calculate 
`output`. In another word, 
   for `y = activation(x)`, it calculates `dx = _backward_activation(dy, y)`, 
not `dx = _backward_activation(dy, x)`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lanking520 commented on a change in pull request #10660: [MXNET-357] New Scala API Design (Symbol)

[GitBox]

lanking520 commented on a change in pull request #10660: [MXNET-357] New Scala 
API Design (Symbol)
URL: https://github.com/apache/incubator-mxnet/pull/10660#discussion_r186820348
 
 

 ##
 File path: scala-package/init/src/main/scala/org/apache/mxnet/init/Base.scala
 ##
 @@ -37,7 +37,11 @@ object Base {
 
   @throws(classOf[UnsatisfiedLinkError])
   private def tryLoadInitLibrary(): Unit = {
-val baseDir = System.getProperty("user.dir") + "/init-native"
+// val baseDir = System.getProperty("user.dir") + "/init-native"
+var baseDir = System.getProperty("user.dir") + "/init-native"
+if (System.getenv().containsKey("BASEDIR")) {
+  baseDir = sys.env("BASEDIR")
+}
 
 Review comment:
   BASEDIR locate in the pom file (set as an environment variable) which 
determine the current base directory.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nswamy commented on a change in pull request #10660: [MXNET-357] New Scala API Design (Symbol)

[GitBox]

nswamy commented on a change in pull request #10660: [MXNET-357] New Scala API 
Design (Symbol)
URL: https://github.com/apache/incubator-mxnet/pull/10660#discussion_r186819525
 
 

 ##
 File path: scala-package/init/src/main/scala/org/apache/mxnet/init/Base.scala
 ##
 @@ -37,7 +37,11 @@ object Base {
 
   @throws(classOf[UnsatisfiedLinkError])
   private def tryLoadInitLibrary(): Unit = {
-val baseDir = System.getProperty("user.dir") + "/init-native"
+// val baseDir = System.getProperty("user.dir") + "/init-native"
+var baseDir = System.getProperty("user.dir") + "/init-native"
+if (System.getenv().containsKey("BASEDIR")) {
+  baseDir = sys.env("BASEDIR")
+}
 
 Review comment:
   what are you expecting BASEDIR variable to be?
   
   I think you should have a else
   `else {
   baseDir System.getProperty("user.dir")
   }
   baseDir = baseDir + "/init-native"
   `
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #9945: Improve dot(csr, row_sparse) on CPU

[GitBox]

eric-haibin-lin commented on issue #9945: Improve dot(csr, row_sparse) on CPU
URL: 
https://github.com/apache/incubator-mxnet/issues/9945#issuecomment-387485755
 
 
   Closing this for now. We probably want to revisit it when we plan to support 
a feature vector of billion dimensions. In that case sort and hash is 
definitely faster than bitmap


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed issue #9945: Improve dot(csr, row_sparse) on CPU

[GitBox]

eric-haibin-lin closed issue #9945: Improve dot(csr, row_sparse) on CPU
URL: https://github.com/apache/incubator-mxnet/issues/9945
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed issue #9921: [DISCUSSION] module.contrib.SparseModule API

[GitBox]

eric-haibin-lin closed issue #9921: [DISCUSSION] module.contrib.SparseModule API
URL: https://github.com/apache/incubator-mxnet/issues/9921
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin closed issue #10509: [Improvement] Distinguish Sparse/Dense operator for temp resource request

[GitBox]

eric-haibin-lin closed issue #10509: [Improvement] Distinguish Sparse/Dense 
operator for temp resource request
URL: https://github.com/apache/incubator-mxnet/issues/10509
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] larroy commented on a change in pull request #10817: [WIP] Do Not Merge. Static memory allocation for cached_op

[GitBox]

larroy commented on a change in pull request #10817: [WIP] Do Not Merge. Static 
memory allocation for cached_op
URL: https://github.com/apache/incubator-mxnet/pull/10817#discussion_r186809871
 
 

 ##
 File path: include/mxnet/op_attr_types.h
 ##
 @@ -145,6 +145,11 @@ class OpStatePtr {
   void reset() {
 ptr_.reset();
   }
+  /* \brief checks whether the managed object is managed only by the current
+OpStatePtr instance */
+  bool unique() {
 
 Review comment:
   where is this used? why expose this detail?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashokei commented on issue #10819: [MXNET-367] update mkldnn to v0.14 and disable building test examples

[GitBox]

ashokei commented on issue #10819: [MXNET-367] update mkldnn to v0.14 and 
disable building test examples
URL: https://github.com/apache/incubator-mxnet/pull/10819#issuecomment-387482207
 
 
   @marcoabreu can you please merge this if ok.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on a change in pull request #10847: [MXNET-408] [WIP] inplace ReLU activation

[GitBox]

eric-haibin-lin commented on a change in pull request #10847: [MXNET-408] [WIP] 
inplace ReLU activation
URL: https://github.com/apache/incubator-mxnet/pull/10847#discussion_r186802962
 
 

 ##
 File path: tests/python/gpu/test_operator_gpu.py
 ##
 @@ -1133,14 +1133,17 @@ def test_fullyconnected_with_type():
 
 @with_seed()
 def test_activation_with_type():
-sym = mx.sym.Activation(name='act', act_type='sigmoid')
-ctx_list = [{'ctx': mx.gpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float64}},
-{'ctx': mx.gpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float32}},
-{'ctx': mx.gpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float16}},
-{'ctx': mx.cpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float64}},
-{'ctx': mx.cpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float32}},
-{'ctx': mx.cpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float16}}]
-check_consistency(sym, ctx_list)
+act_types = ['relu', 'sigmoid', 'tanh', 'softrelu', 'softsign']
+shape = (2, 2, 10, 10)
+for act_type in act_types:
+sym = mx.sym.Activation(name='act', act_type=act_type)
+ctx_list = [{'ctx': mx.gpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float64}},
+{'ctx': mx.gpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float32}},
+{'ctx': mx.gpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float16}},
+{'ctx': mx.cpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float64}},
+{'ctx': mx.cpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float32}},
+{'ctx': mx.cpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float16}}]
+check_consistency(sym, ctx_list)
 
 Review comment:
   Good point. I didn't know it doesn't check grad


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: fix a bug in prepare_mkldnn.sh (#10843)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new e573097  fix a bug in prepare_mkldnn.sh (#10843)
e573097 is described below

commit e57309736935f0b092aa6e27a12a7617fefa778c
Author: Da Zheng 
AuthorDate: Tue May 8 10:07:57 2018 -0700

fix a bug in prepare_mkldnn.sh (#10843)
---
 prepare_mkldnn.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/prepare_mkldnn.sh b/prepare_mkldnn.sh
index 828cfe1..9b11b4a 100755
--- a/prepare_mkldnn.sh
+++ b/prepare_mkldnn.sh
@@ -72,7 +72,7 @@ if [ ! -z "$HOME_MKLDNN" ]; then
   fi
 fi
 
-if [ $OSTYPE == "darwin16" ]; then
+if [ $(uname) == "Darwin" ]; then
   OMP_LIBFILE="$MKLDNN_INSTALLDIR/lib/libiomp5.dylib"
   MKLML_LIBFILE="$MKLDNN_INSTALLDIR/lib/libmklml.dylib"
   MKLDNN_LIBFILE="$MKLDNN_INSTALLDIR/lib/libmkldnn.0.dylib"

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[GitHub] piiswrong closed pull request #10843: fix a bug in prepare_mkldnn.sh for mac

[GitBox]

piiswrong closed pull request #10843: fix a bug in prepare_mkldnn.sh for mac
URL: https://github.com/apache/incubator-mxnet/pull/10843
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/prepare_mkldnn.sh b/prepare_mkldnn.sh
index 828cfe107ed..9b11b4ac6fd 100755
--- a/prepare_mkldnn.sh
+++ b/prepare_mkldnn.sh
@@ -72,7 +72,7 @@ if [ ! -z "$HOME_MKLDNN" ]; then
   fi
 fi
 
-if [ $OSTYPE == "darwin16" ]; then
+if [ $(uname) == "Darwin" ]; then
   OMP_LIBFILE="$MKLDNN_INSTALLDIR/lib/libiomp5.dylib"
   MKLML_LIBFILE="$MKLDNN_INSTALLDIR/lib/libmklml.dylib"
   MKLDNN_LIBFILE="$MKLDNN_INSTALLDIR/lib/libmkldnn.0.dylib"


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #10847: [MXNET-408] [WIP] inplace ReLU activation

[GitBox]

piiswrong commented on a change in pull request #10847: [MXNET-408] [WIP] 
inplace ReLU activation
URL: https://github.com/apache/incubator-mxnet/pull/10847#discussion_r186799392
 
 

 ##
 File path: tests/python/gpu/test_operator_gpu.py
 ##
 @@ -1133,14 +1133,17 @@ def test_fullyconnected_with_type():
 
 @with_seed()
 def test_activation_with_type():
-sym = mx.sym.Activation(name='act', act_type='sigmoid')
-ctx_list = [{'ctx': mx.gpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float64}},
-{'ctx': mx.gpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float32}},
-{'ctx': mx.gpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float16}},
-{'ctx': mx.cpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float64}},
-{'ctx': mx.cpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float32}},
-{'ctx': mx.cpu(0), 'act_data': (2, 2, 10, 10), 'type_dict': 
{'act_data': np.float16}}]
-check_consistency(sym, ctx_list)
+act_types = ['relu', 'sigmoid', 'tanh', 'softrelu', 'softsign']
+shape = (2, 2, 10, 10)
+for act_type in act_types:
+sym = mx.sym.Activation(name='act', act_type=act_type)
+ctx_list = [{'ctx': mx.gpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float64}},
+{'ctx': mx.gpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float32}},
+{'ctx': mx.gpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float16}},
+{'ctx': mx.cpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float64}},
+{'ctx': mx.cpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float32}},
+{'ctx': mx.cpu(0), 'act_data': shape, 'type_dict': 
{'act_data': np.float16}}]
+check_consistency(sym, ctx_list)
 
 Review comment:
   we need to add gradient tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: Finish prerequisites on MXNetTutorialTemplate (#10851)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new b64b9a0  Finish prerequisites on MXNetTutorialTemplate (#10851)
b64b9a0 is described below

commit b64b9a00d57a8a29a1d7ce1f988e03c16b2e9730
Author: Pigeon <32315294+luckypig...@users.noreply.github.com>
AuthorDate: Wed May 9 00:59:35 2018 +0800

Finish prerequisites on MXNetTutorialTemplate (#10851)

* Update MXNetTutorialTemplate.ipynb

*  Add prerequisites URL

give hyper link to each of MXNet, Language, Tool, Familiarity with concept 
or tool.

* Update MXNetTutorialTemplate.ipynb

* Try the symbol title number

use "" and "" instead of "<" and ">"

* try make title number in markdown

use " ` " to solve the " < " disappear problem.

* solve the symbol title number disappear

find all "< >" and replace to "`< >`"

* change some start sign to title number sign

Change some symbols, which sentence is more appropriate to use title number

* change title number

change some title number to star
---
 example/MXNetTutorialTemplate.ipynb | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/example/MXNetTutorialTemplate.ipynb 
b/example/MXNetTutorialTemplate.ipynb
index 2ec9b85..851a87f 100644
--- a/example/MXNetTutorialTemplate.ipynb
+++ b/example/MXNetTutorialTemplate.ipynb
@@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
-"A brief explanation of how the reader can use the tutorial. Can the 
reader copy each code snippet into a Python or other environment? Or can the 
reader run  before or after reading through the explanations to 
understand how the code works?"
+"A brief explanation of how the reader can use the tutorial. Can the 
reader copy each code snippet into a Python or other environment? Or can the 
reader run `` before or after reading through the explanations to 
understand how the code works?"
]
   },
   {
@@ -70,10 +70,10 @@
"source": [
 "To complete this tutorial, you need:\n",
 "\n",
-"- [MXNet](//http://mxnet.io/get_started/setup.html#overview)\n",
-"- [Language](http://)\n",
-"- [Tool](http://)\n",
-"- Familiarity with concept or tool"
+"- [MXNet](https://mxnet.incubator.apache.org/install/#overview)\n",
+"- [Language](https://mxnet.incubator.apache.org/tutorials/)\n",
+"- [Tool](https://mxnet.incubator.apache.org/api/python/index.html)\n",
+"- [Familiarity with concept or tool](https://gluon.mxnet.io/)\n"
]
   },
   {
@@ -96,10 +96,9 @@
"source": [
 "You can download the data used in this tutorial from the [Site 
Name](http://) site. To download the data:\n",
 "\n",
-"1. At the  prompt, type:\n",
-"\n",
-"``command``\n",
+"1. At the `` prompt, type:\n",
 "\n",
+"``\n",
 "2. Second task.\n",
 "\n",
 "3. Last task."
@@ -109,7 +108,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
-"Briefly describe key aspects of the data. If there are two or more 
aspects of the data that require involved discussion, use subheads (### 
). To include a graphic, introduce it with a 
brief description and use the image linking tool to include it. Store the 
graphic in GitHub and use the following format: https://cloud.githubusercontent.com/assets/5545640/15089697/d6f4fca0-13d7-11e6-
 [...]
+"Briefly describe key aspects of the data. If there are two or more 
aspects of the data that require involved discussion, use subheads (### 
``). To include a graphic, introduce it with a 
brief description and use the image linking tool to include it. Store the 
graphic in GitHub and use the following format: https://cloud.githubusercontent.com/assets/5545640/15089697/d6f4fca0-13d7-11e
 [...]
]
   },
   {
@@ -343,7 +342,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
-"To *perform the task*, *provide explanation here.*"
+"To *fperform the task*, *provide explanation here.*"
]
   },
   {

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[GitHub] piiswrong closed pull request #10851: Finish prerequisites on MXNetTutorialTemplate

[GitBox]

piiswrong closed pull request #10851: Finish prerequisites on 
MXNetTutorialTemplate
URL: https://github.com/apache/incubator-mxnet/pull/10851
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/example/MXNetTutorialTemplate.ipynb 
b/example/MXNetTutorialTemplate.ipynb
index 2ec9b8562a4..851a87f1824 100644
--- a/example/MXNetTutorialTemplate.ipynb
+++ b/example/MXNetTutorialTemplate.ipynb
@@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
-"A brief explanation of how the reader can use the tutorial. Can the 
reader copy each code snippet into a Python or other environment? Or can the 
reader run  before or after reading through the explanations to 
understand how the code works?"
+"A brief explanation of how the reader can use the tutorial. Can the 
reader copy each code snippet into a Python or other environment? Or can the 
reader run `` before or after reading through the explanations to 
understand how the code works?"
]
   },
   {
@@ -70,10 +70,10 @@
"source": [
 "To complete this tutorial, you need:\n",
 "\n",
-"- [MXNet](//http://mxnet.io/get_started/setup.html#overview)\n",
-"- [Language](http://)\n",
-"- [Tool](http://)\n",
-"- Familiarity with concept or tool"
+"- [MXNet](https://mxnet.incubator.apache.org/install/#overview)\n",
+"- [Language](https://mxnet.incubator.apache.org/tutorials/)\n",
+"- [Tool](https://mxnet.incubator.apache.org/api/python/index.html)\n",
+"- [Familiarity with concept or tool](https://gluon.mxnet.io/)\n"
]
   },
   {
@@ -96,10 +96,9 @@
"source": [
 "You can download the data used in this tutorial from the [Site 
Name](http://) site. To download the data:\n",
 "\n",
-"1. At the  prompt, type:\n",
-"\n",
-"``command``\n",
+"1. At the `` prompt, type:\n",
 "\n",
+"``\n",
 "2. Second task.\n",
 "\n",
 "3. Last task."
@@ -109,7 +108,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
-"Briefly describe key aspects of the data. If there are two or more 
aspects of the data that require involved discussion, use subheads (### 
). To include a graphic, introduce it with a 
brief description and use the image linking tool to include it. Store the 
graphic in GitHub and use the following format: https://cloud.githubusercontent.com/assets/5545640/15089697/d6f4fca0-13d7-11e6-9331-7f94fcc7b4c6.png\;>.
 You do not need to provide a title for your graphics. "
+"Briefly describe key aspects of the data. If there are two or more 
aspects of the data that require involved discussion, use subheads (### 
``). To include a graphic, introduce it with a 
brief description and use the image linking tool to include it. Store the 
graphic in GitHub and use the following format: https://cloud.githubusercontent.com/assets/5545640/15089697/d6f4fca0-13d7-11e6-9331-7f94fcc7b4c6.png\;>.
 You do not need to provide a title for your graphics. "
]
   },
   {
@@ -343,7 +342,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
-"To *perform the task*, *provide explanation here.*"
+"To *fperform the task*, *provide explanation here.*"
]
   },
   {


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on issue #10852: [MXNET-411] Add ROI Align

[GitBox]

piiswrong commented on issue #10852: [MXNET-411] Add ROI Align
URL: https://github.com/apache/incubator-mxnet/pull/10852#issuecomment-387470994
 
 
   Please add tests and documentation


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #10825: Adding max_area to random_size_crop

[GitBox]

piiswrong commented on a change in pull request #10825: Adding max_area to 
random_size_crop
URL: https://github.com/apache/incubator-mxnet/pull/10825#discussion_r186794091
 
 

 ##
 File path: python/mxnet/image/image.py
 ##
 @@ -457,9 +459,15 @@ def random_size_crop(src, size, min_area, ratio, 
interp=2):
 
 """
 h, w, _ = src.shape
-area = h * w
+src_area = h * w
+
+if 'min_area' in kwargs:
+warnings.warn('`min_area` is deprecated. Please use `area` instead.', 
DeprecationWarning)
+area = kwargs.get('min_area')
 
 Review comment:
   ```
   if 'min_area' in kwargs:
   warnings.warn('`min_area` is deprecated. Please use `area` 
instead.', DeprecationWarning)
   area = kwargs.pop('min_area')
   assert not kwargs, "unexpected keyword arguments ..."
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhanghang1989 opened a new pull request #10852: [MXNET-411] Add ROI Align

[GitBox]

zhanghang1989 opened a new pull request #10852: [MXNET-411] Add ROI Align
URL: https://github.com/apache/incubator-mxnet/pull/10852
 
 
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ ] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments 
are documented. 
   - For new examples, README.md is added to explain the what the example does, 
the source of the dataset, expected performance on test set and reference to 
the original paper if applicable
   - Check the API doc at 
http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [ ] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eric-haibin-lin commented on issue #10835: why SparseEmbedding layer's weight is always dense?

[GitBox]

eric-haibin-lin commented on issue #10835: why SparseEmbedding layer's weight 
is always dense?
URL: 
https://github.com/apache/incubator-mxnet/issues/10835#issuecomment-387465162
 
 
   Lazy initialization for each row in the row_sparse weight parameter is an 
optimization we haven't done yet. You're right - ideally the weight can be 
initialized only when a certain category is seen. Currently all rows are filled 
at one shot during initialization. What sparse embedding provides is the 
ability to only retrieve the parameters for the categories in the current 
mini-batch (instead of loading the full model). 
   For example, you might have a full model on CPU. For each mini-batch, only 
load the rows for the seen categories on GPU and perform forward backward. 
   
   Lazy initialization is definitely worth investigations though. What's your 
use case / application? 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] marcoabreu commented on a change in pull request #10849: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub

[GitBox]

marcoabreu commented on a change in pull request #10849: [MXNET-409] Temporary 
fix for ARM64 builds - switch to own dockerhub
URL: https://github.com/apache/incubator-mxnet/pull/10849#discussion_r186777544
 
 

 ##
 File path: ci/docker/Dockerfile.build.arm64
 ##
 @@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 Review comment:
   I'm indifferent to that


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dwSun commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() (5 vs. 5)

[GitBox]

dwSun commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() 
(5 vs. 5)
URL: 
https://github.com/apache/incubator-mxnet/issues/10809#issuecomment-387420123
 
 
   tested with mxnet-mkl-1.2.0b20180508, fashion.py in this issue works well.
   But with mxnet-cu91mkl-1.2.0b20180507, I got this error:
   ```
   [14:53:41] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running 
performance tests to find the best convolution algorithm, this can take a 
while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   Stack trace returned 10 entries:
   [bt] (0) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2ec7b2)
 [0x7f1db68a87b2]
   [bt] (1) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2dafa1e)
 [0x7f1db936ba1e]
   [bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f1df499ff20]
   [bt] (3) /lib64/ld-linux-x86-64.so.2(+0xcac8) [0x7f1df5b71ac8]
   [bt] (4) /lib64/ld-linux-x86-64.so.2(+0x150bd) [0x7f1df5b7a0bd]
   [bt] (5) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f1df4ac82df]
   [bt] (6) /lib64/ld-linux-x86-64.so.2(+0x147ca) [0x7f1df5b797ca]
   [bt] (7) /lib/x86_64-linux-gnu/libc.so.6(+0x1663ad) [0x7f1df4ac73ad]
   [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f1df4ac82df]
   [bt] (9) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x2f) 
[0x7f1df4ac836f]
   
   ```
   I am using ubuntu18.04 and cuda9.1 installed with:
   ```sh
   sudo aptitude install nvidia-cuda-toolkit --without-recommends
   ```
   
   should I start a new issue?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dwSun commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() (5 vs. 5)

[GitBox]

dwSun commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() 
(5 vs. 5)
URL: 
https://github.com/apache/incubator-mxnet/issues/10809#issuecomment-387420123
 
 
   tested with mxnet-mkl-1.2.0b20180508, fashion.py in this issue works well.
   But with mxnet-cu91mkl-1.2.0b20180507, I got this error:
   ```
   [14:52:48] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running 
performance tests to find the best convolution algorithm, this can take a 
while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   Stack trace returned 10 entries:
   [bt] (0) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2ec7b2)
 [0x7f9f9bc6a7b2]
   [bt] (1) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2dafa1e)
 [0x7f9f9e72da1e]
   [bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f9fd9d61f20]
   [bt] (3) /lib64/ld-linux-x86-64.so.2(+0xcac8) [0x7f9fdaf33ac8]
   [bt] (4) /lib64/ld-linux-x86-64.so.2(+0x150bd) [0x7f9fdaf3c0bd]
   [bt] (5) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f9fd9e8a2df]
   [bt] (6) /lib64/ld-linux-x86-64.so.2(+0x147ca) [0x7f9fdaf3b7ca]
   [bt] (7) /lib/x86_64-linux-gnu/libc.so.6(+0x1663ad) [0x7f9fd9e893ad]
   [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f9fd9e8a2df]
   [bt] (9) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x2f) 
[0x7f9fd9e8a36f]
   
   Segmentation fault: 11
   
   Stack trace returned 10 entries:
   [bt] (0) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2ec7b2)
 [0x7f9f9bc6a7b2]
   [bt] (1) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2dafa1e)
 [0x7f9f9e72da1e]
   [bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f9fd9d61f20]
   [bt] (3) /lib64/ld-linux-x86-64.so.2(+0xcac8) [0x7f9fdaf33ac8]
   [bt] (4) /lib64/ld-linux-x86-64.so.2(+0x150bd) [0x7f9fdaf3c0bd]
   [bt] (5) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f9fd9e8a2df]
   [bt] (6) /lib64/ld-linux-x86-64.so.2(+0x147ca) [0x7f9fdaf3b7ca]
   [bt] (7) /lib/x86_64-linux-gnu/libc.so.6(+0x1663ad) [0x7f9fd9e893ad]
   [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f9fd9e8a2df]
   [bt] (9) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x2f) 
[0x7f9fd9e8a36f]
   (mkl-dnn)  ✘ david@MiniNuke  ~/face  python3 fashion.py 
   [14:53:41] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running 
performance tests to find the best convolution algorithm, this can take a 
while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   
   Segmentation fault: 11
   
   Stack trace returned 10 entries:
   [bt] (0) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2ec7b2)
 [0x7f1db68a87b2]
   [bt] (1) 
/home/david/.virtualenvs/mkl-dnn/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2dafa1e)
 [0x7f1db936ba1e]
   [bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f1df499ff20]
   [bt] (3) /lib64/ld-linux-x86-64.so.2(+0xcac8) [0x7f1df5b71ac8]
   [bt] (4) /lib64/ld-linux-x86-64.so.2(+0x150bd) [0x7f1df5b7a0bd]
   [bt] (5) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f1df4ac82df]
   [bt] (6) /lib64/ld-linux-x86-64.so.2(+0x147ca) [0x7f1df5b797ca]
   [bt] (7) /lib/x86_64-linux-gnu/libc.so.6(+0x1663ad) [0x7f1df4ac73ad]
   [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x6f) 
[0x7f1df4ac82df]
   [bt] (9) /lib/x86_64-linux-gnu/libc.so.6(_dl_catch_error+0x2f) 
[0x7f1df4ac836f]
   
   ```
   I am using ubuntu18.04 and cuda9.1 installed with:
   ```sh
   sudo aptitude install nvidia-cuda-toolkit --without-recommends
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] TaoLv commented on issue #10104: [WIP][MXNET-107] Fused RNN implementation for CPU

[GitBox]

TaoLv commented on issue #10104: [WIP][MXNET-107] Fused RNN implementation for 
CPU
URL: https://github.com/apache/incubator-mxnet/pull/10104#issuecomment-387425685
 
 
   I feel it difficult to change the existing gluon LSTM layer from normal 
`Block` to `HybridBlock` without changing APIs.
   (1) I need concatenate the exsiting `i2h_weight`, `h2h_weight`, `i2h_bias` 
and `h2h_bias` together to feed them into the fused operator. I think that is 
time consuming. 
[link](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/rnn/rnn_layer.py#L62)
   (2) I cannot create `begin_state` if it's not presented in the 
`hybrid_forward` function, since I cannot get the shape and batch size here in 
a HybridBlock. 
[link](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/rnn/rnn_layer.py#L174)
   Maybe I missed something. Any cues about it? @szha @piiswrong 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dwSun commented on issue #10580: inference results unstable in mxnet_mkl-1.2.0b20180416

[GitBox]

dwSun commented on issue #10580: inference results unstable in 
mxnet_mkl-1.2.0b20180416 
URL: 
https://github.com/apache/incubator-mxnet/issues/10580#issuecomment-387420798
 
 
   just tested with mxnet-mkl-1.2.0b20180508, it works well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dwSun commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() (5 vs. 5)

[GitBox]

dwSun commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() 
(5 vs. 5)
URL: 
https://github.com/apache/incubator-mxnet/issues/10809#issuecomment-387420123
 
 
   tested with mxnet-mkl-1.2.0b20180508, it works well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] LuckyPigeon opened a new pull request #10851: Finish prerequisites on MXNetTutorialTemplate

[GitBox]

LuckyPigeon opened a new pull request #10851: Finish prerequisites on 
MXNetTutorialTemplate
URL: https://github.com/apache/incubator-mxnet/pull/10851
 
 
   ## Description ##
   (Brief description on what this PR is about)
   JIRA issue: MXNET-410
   Finish prerequisites's hyper links and and fix the problem "< >" will 
disappear in markdown.
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [v ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to 
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) 
created (except PRs with tiny changes)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [v ] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding 
a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing 
distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a 
new build option with NCCL)
   - [ v] To the my best knowledge, examples are either not affected by this 
change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [v ] Feature1, tests, (and when applicable, API doc)
   - [v ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Interesting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pengzhao-intel commented on issue #10809: Check failed: format != mkl_mem_->GetFormat() (5 vs. 5)

[GitBox]

pengzhao-intel commented on issue #10809: Check failed: format != 
mkl_mem_->GetFormat() (5 vs. 5)
URL: 
https://github.com/apache/incubator-mxnet/issues/10809#issuecomment-387414766
 
 
   @dwSun, Tao's PR  is merged. 
   Could you try the case again?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pengzhao-intel commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory and then crash down.

[GitBox]

pengzhao-intel commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of 
memory and then crash down.
URL: 
https://github.com/apache/incubator-mxnet/issues/10599#issuecomment-387413987
 
 
   @al-rigazzi could you share the script with us? 
   Did you build the code in the Haswell machine as well?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] CodingCat commented on a change in pull request #10462: [MXNET-62] add test against spark integration

[GitBox]

CodingCat commented on a change in pull request #10462: [MXNET-62] add test 
against spark integration
URL: https://github.com/apache/incubator-mxnet/pull/10462#discussion_r186738641
 
 

 ##
 File path: 
scala-package/spark/src/test/scala/org/apache/mxnet/spark/MXNetGeneralSuite.scala
 ##
 @@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.mxnet.spark
+
+import org.apache.spark.SparkContext
+import org.apache.spark.mllib.linalg.Vectors
+import org.apache.spark.mllib.regression.LabeledPoint
+import org.apache.spark.rdd.RDD
+
+class MXNetGeneralSuite extends SharedSparkContext {
+
+  private def parseRawData(sc: SparkContext, path: String): RDD[LabeledPoint] 
= {
+val raw = sc.textFile(path)
+raw.map { s =>
+  val parts = s.split(' ')
+  val label = java.lang.Double.parseDouble(parts(0))
+  val features = 
Vectors.dense(parts(1).trim().split(',').map(java.lang.Double.parseDouble))
+  LabeledPoint(label, features)
+}
+  }
+
+  test("run spark with MLP") {
+val trainData = parseRawData(sc,
+  "/Users/nanzhu/code/mxnet/scala-package/spark/train.txt")
 
 Review comment:
   @szha @piiswrong , thanks ! I have uploaded file at 
https://github.com/dmlc/web-data/pull/63


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] al-rigazzi commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory and then crash down.

[GitBox]

al-rigazzi commented on issue #10599: [MKLDNN Bug] MKLDNN eats lots of memory 
and then crash down.
URL: 
https://github.com/apache/incubator-mxnet/issues/10599#issuecomment-387411908
 
 
   On a 24-core Haswell, every Resnet v1 network crashes at the first step.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg commented on a change in pull request #10849: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub

[GitBox]

lebeg commented on a change in pull request #10849: [MXNET-409] Temporary fix 
for ARM64 builds - switch to own dockerhub
URL: https://github.com/apache/incubator-mxnet/pull/10849#discussion_r186729826
 
 

 ##
 File path: ci/docker/Dockerfile.build.arm64
 ##
 @@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 Review comment:
   Maybe ```FROM 
dockcross/linux-arm64@sha256:4b89d8ff7a039fc0886e8ed06d82b82cd962dfe00db664efbba544ed9002f9b0```
 would be more simple for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg commented on a change in pull request #10850: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub (1.2.0)

[GitBox]

lebeg commented on a change in pull request #10850: [MXNET-409] Temporary fix 
for ARM64 builds - switch to own dockerhub (1.2.0)
URL: https://github.com/apache/incubator-mxnet/pull/10850#discussion_r186730022
 
 

 ##
 File path: ci/docker/Dockerfile.build.arm64
 ##
 @@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 Review comment:
   Maybe ```FROM 
dockcross/linux-arm64@sha256:4b89d8ff7a039fc0886e8ed06d82b82cd962dfe00db664efbba544ed9002f9b0```
 would be more simple for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg commented on a change in pull request #10850: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub (1.2.0)

[GitBox]

lebeg commented on a change in pull request #10850: [MXNET-409] Temporary fix 
for ARM64 builds - switch to own dockerhub (1.2.0)
URL: https://github.com/apache/incubator-mxnet/pull/10850#discussion_r186730022
 
 

 ##
 File path: ci/docker/Dockerfile.build.arm64
 ##
 @@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 Review comment:
   Maybe ```FROM dockcross/linux-arm64@sha256: 
6191491ba51a949eadbfd842a1352699f5ca8489``` would be more simple for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg commented on a change in pull request #10850: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub (1.2.0)

[GitBox]

lebeg commented on a change in pull request #10850: [MXNET-409] Temporary fix 
for ARM64 builds - switch to own dockerhub (1.2.0)
URL: https://github.com/apache/incubator-mxnet/pull/10850#discussion_r186730022
 
 

 ##
 File path: ci/docker/Dockerfile.build.arm64
 ##
 @@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 Review comment:
   Maybe ```FROM 
dockcross/linux-arm64@sha256:6191491ba51a949eadbfd842a1352699f5ca8489``` would 
be more simple for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg commented on a change in pull request #10849: [MXNET-409] Temporary fix for ARM64 builds - switch to own dockerhub

[GitBox]

lebeg commented on a change in pull request #10849: [MXNET-409] Temporary fix 
for ARM64 builds - switch to own dockerhub
URL: https://github.com/apache/incubator-mxnet/pull/10849#discussion_r186729826
 
 

 ##
 File path: ci/docker/Dockerfile.build.arm64
 ##
 @@ -18,21 +18,22 @@
 #
 # Dockerfile to build MXNet for ARM64/ARMv8
 
-FROM dockcross/linux-arm64
+# Temporary fix due to https://github.com/apache/incubator-mxnet/issues/10837
+#FROM dockcross/linux-arm64
+FROM mxnetci/dockcross-linux-arm64:05082018
 
 Review comment:
   Maybe ```FROM dockcross/linux-arm64@sha256: 
6191491ba51a949eadbfd842a1352699f5ca8489``` would be more simple for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jacksonloper commented on issue #10101: gluon feature request: proper registration/initialization of layers inside a list (container) for custom (Hybrid)Blocks

[GitBox]

jacksonloper commented on issue #10101: gluon feature request: proper 
registration/initialization of layers inside a list (container) for custom 
(Hybrid)Blocks
URL: 
https://github.com/apache/incubator-mxnet/issues/10101#issuecomment-387404132
 
 
   This solution is kind of weird.  Sequential feels like it ought to be 
composed of things that can feed into one another.  But if you are just using 
it as a list, the shapes might not even be right for that.
   
   I admit it isn't a high priority, but just for sugar it might be nice to 
implement a separate blocklist class.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg commented on issue #10848: Fixed armv8 build

[GitBox]

lebeg commented on issue #10848: Fixed armv8 build
URL: https://github.com/apache/incubator-mxnet/pull/10848#issuecomment-387400632
 
 
   Fix is incorporated in https://github.com/apache/incubator-mxnet/pull/10850 
and https://github.com/apache/incubator-mxnet/pull/10849


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lebeg closed pull request #10848: Fixed armv8 build

[GitBox]

lebeg closed pull request #10848: Fixed armv8 build
URL: https://github.com/apache/incubator-mxnet/pull/10848
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/ci/docker/Dockerfile.build.arm64 b/ci/docker/Dockerfile.build.arm64
index eb68a818ba6..5ee59b12c04 100755
--- a/ci/docker/Dockerfile.build.arm64
+++ b/ci/docker/Dockerfile.build.arm64
@@ -21,18 +21,16 @@
 FROM dockcross/linux-arm64
 
 ENV ARCH aarch64
-ENV CC /usr/bin/aarch64-linux-gnu-gcc
-ENV CXX /usr/bin/aarch64-linux-gnu-g++
-ENV FC /usr/bin/aarch64-linux-gnu-gfortran-4.9
 ENV HOSTCC gcc
+ENV TARGET ARMV8
 
-WORKDIR /work
+WORKDIR /work/deps
 
-COPY install/arm64_openblas.sh /work/
-RUN /work/arm64_openblas.sh
+# Build OpenBLAS
+RUN git clone --recursive -b v0.2.20 https://github.com/xianyi/OpenBLAS.git && 
\
+cd OpenBLAS && \
+make -j$(nproc) && \
+make PREFIX=$CROSS_ROOT install
 
-ENV LD_LIBRARY_PATH /opt/OpenBLAS/lib
-ENV CPLUS_INCLUDE_PATH /opt/OpenBLAS/include
+COPY runtime_functions.sh /work/
 WORKDIR /work/mxnet
-
-COPY runtime_functions.sh /work/
\ No newline at end of file


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services