[GitHub] dabraude commented on issue #10261: [MXNET-128] added load from buffer functions
dabraude commented on issue #10261: [MXNET-128] added load from buffer functions URL: https://github.com/apache/incubator-mxnet/pull/10261#issuecomment-378145556 Hey @cjolivier01 I was wondering if I needed to do something else to get this merged? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nju-luke commented on issue #10368: asscalar is very slow
nju-luke commented on issue #10368: asscalar is very slow URL: https://github.com/apache/incubator-mxnet/issues/10368#issuecomment-378137784 The OOM was fixed by changed train_loss with `train_loss += nd.mean(loss_).asscalar()` . Appreciations @reminisce This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nju-luke commented on issue #10368: asscalar is very slow
nju-luke commented on issue #10368: asscalar is very slow URL: https://github.com/apache/incubator-mxnet/issues/10368#issuecomment-378137784 The OOM was fixed by changing train_loss with `train_loss += nd.mean(loss_).asscalar()` . Appreciations @reminisce This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nju-luke closed issue #10368: asscalar is very slow
nju-luke closed issue #10368: asscalar is very slow URL: https://github.com/apache/incubator-mxnet/issues/10368 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] TaoLv commented on issue #10316: MultiBoxDetection cannot pass consistency check
TaoLv commented on issue #10316: MultiBoxDetection cannot pass consistency check URL: https://github.com/apache/incubator-mxnet/issues/10316#issuecomment-378132736 There is an `atomicAdd` in the [cuda implementation](https://github.com/apache/incubator-mxnet/blob/master/src/operator/contrib/multibox_detection.cu#L93) of MultiBoxDetection. It will generate unstable results which are not reproducible. So I cannot check the consistency between cpu and gpu computation. I am not sure if it's as expectation. But I think you should be aware of this. @zhreshold @piiswrong This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] anirudh2290 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU
anirudh2290 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU URL: https://github.com/apache/incubator-mxnet/pull/10371#issuecomment-378058175 dot(dns, csr) output a csr ndarray when cpu context is used, and dns ndarray when gpu context is used ? This should be at least well documented somewhere. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test
xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test URL: https://github.com/apache/incubator-mxnet/pull/10365#issuecomment-378125832 I think the failure of this ut test may be related to this old version of mklml. https://github.com/apache/incubator-mxnet/blob/5245ef68191a6d47594bf331ec6e20ba6e93ad4c/ci/docker/install/ubuntu_mklml.sh#L24 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test
xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test URL: https://github.com/apache/incubator-mxnet/pull/10365#issuecomment-378125832 I think the failure of this ut test may be related to this old version of mkl-dnn. https://github.com/apache/incubator-mxnet/blob/5245ef68191a6d47594bf331ec6e20ba6e93ad4c/ci/docker/install/ubuntu_mklml.sh#L24 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test
xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test URL: https://github.com/apache/incubator-mxnet/pull/10365#issuecomment-378132307 @zheng-da yes, I've made a mistake. It's mklml not mkldnn:) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] TaoLv commented on a change in pull request #10365: [MXNET-261]Update MKLDNN & Add CPP Test
TaoLv commented on a change in pull request #10365: [MXNET-261]Update MKLDNN &
Add CPP Test
URL: https://github.com/apache/incubator-mxnet/pull/10365#discussion_r178716118
##
File path: tests/cpp/operator/mkldnn.cc
##
@@ -77,4 +77,11 @@ TEST(MKLDNN_UTIL_FUNC, AlignMem) {
LOG(INFO) << "Skipped for GCC " << __GNUC__ << "." << __GNUC_MINOR__;
#endif
}
+
+TEST(MKLDNN_UTIL_FUNC, MemFormat) {
+ // Check whether the number of format is correct.
+ CHECK_EQ(mkldnn_format_last, 56);
Review comment:
Yes, it's an enum. This request is from @marcoabreu. Please refer
https://github.com/apache/incubator-mxnet/pull/9918#discussion_r171472242 and
[jira#98](https://issues.apache.org/jira/browse/MXNET-98) for the background of
this. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] anirudhacharya commented on issue #10298: Mxnet not loading
anirudhacharya commented on issue #10298: Mxnet not loading
URL:
https://github.com/apache/incubator-mxnet/issues/10298#issuecomment-378130716
@FinScience What is the MXNet version you are using?
in the meantime, try the following -
```
install.packages("devtools")
require(devtools)
install_version("DiagrammeR", version = "0.9.0", repos =
"http://cran.us.r-project.org";)
require(DiagrammeR)
library("mxnet")
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] anirudhacharya commented on issue #10298: Mxnet not loading
anirudhacharya commented on issue #10298: Mxnet not loading
URL:
https://github.com/apache/incubator-mxnet/issues/10298#issuecomment-378130716
@FinScience What is the MXNet version you are using?
in the meantime, try the following and let me know if it works -
```
install.packages("devtools")
require(devtools)
install_version("DiagrammeR", version = "0.9.0", repos =
"http://cran.us.r-project.org";)
require(DiagrammeR)
library("mxnet")
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] zheng-da commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test
zheng-da commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test URL: https://github.com/apache/incubator-mxnet/pull/10365#issuecomment-378129789 @xinyu-intel I guess you mean mklml? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test
xinyu-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test URL: https://github.com/apache/incubator-mxnet/pull/10365#issuecomment-378125832 I think the failure of this ut test may be related to this old version of mkl-dnn. https://github.com/apache/incubator-mxnet/blob/5245ef68191a6d47594bf331ec6e20ba6e93ad4c/ci/docker/install/ubuntu_mklml.sh#L24 How to build and test with mkl-dnn in 3rdparty? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] moveforever commented on issue #10310: MemoryError on linear classification with 400million dimension feature input
moveforever commented on issue #10310: MemoryError on linear classification
with 400million dimension feature input
URL:
https://github.com/apache/incubator-mxnet/issues/10310#issuecomment-378109878
I came across with same error. The reason of the error is to save
intermediate optimization statement variable while saving checkpoint.
you can fix like this:
mod.save_checkpoint("checkpoint", epoch, save_optimizer_states=False)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] asitstands commented on issue #10367: [MXNET-262] Implement mx.random.seed_context to seed random number generators of a specific device context
asitstands commented on issue #10367: [MXNET-262] Implement mx.random.seed_context to seed random number generators of a specific device context URL: https://github.com/apache/incubator-mxnet/pull/10367#issuecomment-378108174 @piiswrong Omitting `ctx` argument usually means that it is the current context. So I think that adding `ctx=None` to `seed` could be potentially confusing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asitstands commented on issue #10367: [MXNET-262] Implement mx.random.seed_context to seed random number generators of a specific device context
asitstands commented on issue #10367: [MXNET-262] Implement mx.random.seed_context to seed random number generators of a specific device context URL: https://github.com/apache/incubator-mxnet/pull/10367#issuecomment-378108174 @piiswrong Omiting `ctx` argument usually means that it is the current context. So I think that adding `ctx=None` to `seed` could be potentially confusing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] HeliWang opened a new issue #10378: Inconsistent output from mxnet-python and mxnet-scala
HeliWang opened a new issue #10378: Inconsistent output from mxnet-python and
mxnet-scala
URL: https://github.com/apache/incubator-mxnet/issues/10378
## Description
Inconsistent output from mxnet-python and mxnet-scala when importing the
same mxnet saved model (kim-0100.params, kim-symbol.json) and have the same
input (np.ones((1000, 47, 300))).
MxNet (Python) Output:
```
Shape of output:
(1000,6)
First three rows of data:
-38.8981 -3.1511 -13.0952 17.5691 -20.4780 19.5707
-38.8981 -3.1511 -13.0952 17.5691 -20.4780 19.5707
-38.8981 -3.1511 -13.0952 17.5691 -20.4780 19.5707
```
MxNet (Scala) Output:
```
Shape of output:
(1000,6)
First three rows of data:
-35.025154 -5.7320843 -13.143161 17.261858 -19.199905 22.68964
-40.023407 0.13218212 -8.405195 16.911377 -23.552662 14.029578
-39.91144 -8.177831 -6.575444 12.67445 -11.054529 15.726346
```
## Environment info (Required)
```
--Python Info--
('Version :', '2.7.12')
('Compiler :', 'GCC 5.4.0 20160609')
('Build:', ('default', 'Dec 4 2017 14:50:18'))
('Arch :', ('64bit', 'ELF'))
Pip Info---
('Version :', '9.0.3')
('Directory:', '/home/h379wang/.local/lib/python2.7/site-packages/pip')
--MXNet Info---
('Version :', '1.2.0')
('Directory:', '/home/h379wang/incubator-mxnet/python/mxnet')
Hashtag not found. Not installed from pre-built package.
--System Info--
('Platform :', 'Linux-4.13.0-1011-gcp-x86_64-with-Ubuntu-16.04-xenial')
('system :', 'Linux')
('node :', 'instance-1')
('release :', '4.13.0-1011-gcp')
('version :', '#15-Ubuntu SMP Mon Feb 12 16:29:04 UTC 2018')
--Hardware Info--
('machine :', 'x86_64')
('processor:', 'x86_64')
Architecture: x86_64
CPU op-mode(s):32-bit, 64-bit
Byte Order:Little Endian
CPU(s):2
On-line CPU(s) list: 0,1
Thread(s) per core:2
Core(s) per socket:1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family:6
Model: 63
Model name:Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping: 0
CPU MHz: 2300.000
BogoMIPS: 4600.00
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 46080K
NUMA node0 CPU(s): 0,1
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm
constant_tsc rep_good nopl xtopology nonstop_tsc cpuid pni pclmulqdq ssse3 fma
cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand
hypervisor lahf_lm abm invpcid_single pti retpoline fsgsbase tsc_adjust bmi1
avx2 smep bmi2 erms invpcid xsaveopt
--Network Test--
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0073
sec, LOAD: 0.4301 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0016 sec, LOAD:
0.0566 sec.
Timing for FashionMNIST:
https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz,
DNS: 0.0743 sec, LOAD: 0.6235 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0209 sec,
LOAD: 0.0855 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0777 sec, LOAD:
0.4415 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.1679 sec, LOAD:
0.9276 sec.
```
Package used (Python/R/Scala/Julia): Scala and Python
For Scala user, please provide:
1. Java version: (`java -version`)
```
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
```
2. Maven version: (`mvn -version`)
```
Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297;
2018-02-24T14:49:05-05:00)
Maven home: /opt/apache-maven-3.5.3
Java version: 1.8.0_161, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.13.0-1011-gcp", arch: "amd64", family: "unix"
```
3. Scala runtime if applicable: (`scala -version`)
```
Scala code runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL
```
## Build info (Required if built from source)
Compiler (gcc/clang/mingw/visual studio): gcc
MXNet commit hash: 22908431b23538a8f0145817016460b3dd2300f1
(Paste the output of `git rev-parse HEAD` here.)
Build config:
(Paste the content of config.mk, or the build command.)
```
# Li
[GitHub] eric-haibin-lin commented on a change in pull request #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU
eric-haibin-lin commented on a change in pull request #10371: [MXNET-263] [WIP]
Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU
URL: https://github.com/apache/incubator-mxnet/pull/10371#discussion_r178693861
##
File path: src/operator/tensor/dot-inl.h
##
@@ -235,13 +235,21 @@ inline bool DotForwardInferStorageType(const
nnvm::NodeAttrs& attrs,
DispatchMode::kFComputeEx);
}
if (!dispatched && lhs_stype == kDefaultStorage && rhs_stype == kCSRStorage
&&
- !param.transpose_a && !param.transpose_b) {
Review comment:
I thought we will add a stype hint argument which defaults to None?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] TaoLv commented on a change in pull request #10315: [MXNET-249] Add inplace support to mkldnn sum
TaoLv commented on a change in pull request #10315: [MXNET-249] Add inplace
support to mkldnn sum
URL: https://github.com/apache/incubator-mxnet/pull/10315#discussion_r178693603
##
File path: src/operator/nn/mkldnn/mkldnn_sum.cc
##
@@ -49,23 +49,42 @@ void Sum(const mkldnn::memory &arr1, const mkldnn::memory
&arr2,
void MKLDNNSumForward(const nnvm::NodeAttrs& attrs, const OpContext &ctx,
const std::vector &inputs, const OpReqType &req,
const NDArray &out_data) {
+ if (req == kNullOp) {
+return;
+ }
+
TmpMemMgr::Get()->Init(ctx.requested[0]);
std::vector in_prims;
std::vector in_pds(inputs.size());
std::vector scales(inputs.size(), 1);
in_prims.reserve(inputs.size());
+ bool pd_same = true;
for (size_t i = 0; i < inputs.size(); i++) {
auto in_mem = inputs[i].GetMKLDNNData();
in_prims.push_back(*in_mem);
in_pds[i] = in_mem->get_primitive_desc();
+// pd_same = pd_same && (in_pds[i] == in_pds[0]);
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sxjscience commented on issue #9881: Inconsistent weight decay logics in multiple optimizers
sxjscience commented on issue #9881: Inconsistent weight decay logics in multiple optimizers URL: https://github.com/apache/incubator-mxnet/issues/9881#issuecomment-378099199 Looks good. We can write math formulas instead. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sxjscience commented on issue #10041: Reduce operators do not support axis=None
sxjscience commented on issue #10041: Reduce operators do not support axis=None URL: https://github.com/apache/incubator-mxnet/issues/10041#issuecomment-378098968 `None` should represent "empty-axis" and should perform a global reduction. It's very common in numpy. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] fedorzh commented on issue #9410: Training with the same parameters and seed gets significantly different results
fedorzh commented on issue #9410: Training with the same parameters and seed gets significantly different results URL: https://github.com/apache/incubator-mxnet/issues/9410#issuecomment-378098687 @cjolivier01 The function `convert_gluon_dataset_to_numpy` which converts gluon dataset to numpy takes awhile. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] solin319 commented on issue #10366: fix bug in sgd
solin319 commented on issue #10366: fix bug in sgd URL: https://github.com/apache/incubator-mxnet/pull/10366#issuecomment-378098403 @eric-haibin-lin MXNET_EXEC_NUM_TEMP doesn't work. But make MXNET_CPU_TEMP_COPY and MXNET_GPU_TEMP_COPY larger can solve the overlap problem. It's difficult to find the description for these two env vars, and I think it's difficult to use for many users. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] anirudh2290 commented on a change in pull request #10374: Sparse support for Custom Op
anirudh2290 commented on a change in pull request #10374: Sparse support for
Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178691985
##
File path: src/operator/custom/custom.cc
##
@@ -266,97 +267,237 @@ OpStatePtr CreateState(const NodeAttrs& attrs, Context
ctx,
return OpStatePtr::Create(state);
}
-void Forward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Forward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
std::vector ptrs;
std::vector tags;
std::vector cpys;
+ std::unordered_set input_tags({0, 4});
+ std::unordered_set output_tags({1});
auto dev_id = ctx.run_ctx.ctx.dev_id;
for (size_t i = 0; i < params.num_args; ++i) {
-NDArray *nd = new NDArray(inputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(0);
}
for (size_t i = 0; i < params.num_outs; ++i) {
-NDArray *nd = new NDArray(outputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, outputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(1);
}
for (size_t i = 0; i < params.num_auxs; ++i) {
-NDArray *nd = new NDArray(inputs[i+params.num_args], dev_id);
+size_t idx = i + params.num_args;
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, idx, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(4);
}
CustomOperator::Get()->Push(
-[=]() {
-
CHECK(reinterpret_cast(params.info->callbacks[kCustomOpForward])(
-ptrs.size(), const_cast(ptrs.data()),
const_cast(tags.data()),
-reinterpret_cast(req.data()),
static_cast(ctx.is_train),
-params.info->contexts[kCustomOpForward]));
-}, ctx, false, ctx.is_train, cpys);
+ [=]() {
+CHECK(reinterpret_cast(
+params.info->callbacks[kCustomOpForward])(
+ptrs.size(), const_cast(ptrs.data()),
+const_cast(tags.data()),
+reinterpret_cast(req.data()),
+static_cast(ctx.is_train),
+params.info->contexts[kCustomOpForward]));
+ },
+ ctx, false, ctx.is_train, cpys, tags, input_tags, output_tags, inputs,
outputs);
}
-
-void Backward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Backward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
- size_t total = 2*params.num_args + 2*params.num_outs + params.num_auxs;
- std::vector ptrs(params.num_args + 2*params.num_outs, nullptr);
+ size_t total = 2 * params.num_args + 2 * params.num_outs + params.num_auxs;
+ std::vector ptrs(params.num_args + 2 * params.num_outs, nullptr);
+
std::vector tags;
std::vector cpys;
ptrs.reserve(total);
tags.reserve(total);
+ cpys.reserve(total);
+
+ std::unordered_set input_tags({3, 0, 1, 4});
+ std::unordered_set output_tags({2});
+
for (size_t i = 0; i < params.num_outs; ++i) tags.push_back(3);
for (size_t i = 0; i < params.num_args; ++i) tags.push_back(0);
for (size_t i = 0; i < params.num_outs; ++i) tags.push_back(1);
auto dev_id = ctx.run_ctx.ctx.dev_id;
+
for (size_t i = 0; i < params.bwd_idx.size(); ++i) {
-NDArray *nd = new NDArray(inputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, i, dev_id);
cpys.push_back(*nd);
ptrs[params.bwd_idx[i]] = reinterpret_cast(nd);
}
for (size_t i = 0; i < ptrs.size(); ++i) {
-if (ptrs[i] == nullptr) ptrs[i] = reinterpret_cast(new NDArray());
+NDArray* nd;
+if (ptrs[i] == nullptr) {
+nd = new NDArray();
+ptrs[i] = reinterpret_cast(nd);
+}
}
- for (const auto& i : outputs) {
-NDArray* nd = new NDArray(i, dev_id);
+ for (size_t i = 0; i < outputs.size(); ++i) {
+NDArray* nd;
+allocate_ndarray_copy(&nd, outputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(2);
}
+
for (size_t i = 0; i < params.num_auxs; ++i) {
-NDArray* nd = new NDArray(inputs[inputs.size()-params.num_auxs+i], dev_id);
+size_t idx = inputs.size() - params.num_auxs + i;
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, idx, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(4);
}
-
CustomOperator::Get()->Push(
[=]() {
CHECK(reinterpre
[GitHub] eric-haibin-lin commented on issue #9881: Inconsistent weight decay logics in multiple optimizers
eric-haibin-lin commented on issue #9881: Inconsistent weight decay logics in multiple optimizers URL: https://github.com/apache/incubator-mxnet/issues/9881#issuecomment-378096575 @szhengac thanks for your inputs. My concern is that if we change the wd behavior now, the change is incompatible with previous versions of MXNet and users have to change their code for new hyperparameters. I don't think this is provides a good user experience. @szha @piiswrong @szhengac What about specifically documenting the update rules for the existing optimizers like https://mxnet.incubator.apache.org/versions/master/api/python/optimization/optimization.html#mxnet.optimizer.SGD ? For new optimizers, during code review committers should check if the implementation is similar to pytorch/tf ones? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #8914: The custom operator not supported for group context?
eric-haibin-lin commented on issue #8914: The custom operator not supported for group context? URL: https://github.com/apache/incubator-mxnet/issues/8914#issuecomment-378095174 Verified on my end that the fix works. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin closed issue #8914: The custom operator not supported for group context?
eric-haibin-lin closed issue #8914: The custom operator not supported for group context? URL: https://github.com/apache/incubator-mxnet/issues/8914 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #10041: Reduce operators do not support axis=None
eric-haibin-lin commented on issue #10041: Reduce operators do not support axis=None URL: https://github.com/apache/incubator-mxnet/issues/10041#issuecomment-378094996 `b = mx.nd.sum(mx.nd.ones((100, 100))` will work.. I guess the problem is that if user provides axis=None, it won't work. @sxjscience Any specific reason why user would provide None??? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #10041: Reduce operators do not support axis=None
eric-haibin-lin commented on issue #10041: Reduce operators do not support axis=None URL: https://github.com/apache/incubator-mxnet/issues/10041#issuecomment-378094996 `b = mx.nd.sum(mx.nd.ones((100, 100))` will work.. I guess the problem is that if user provides axis=None, it won't work. @sxjscience This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #2317: (info.type) != (kNotInitialized)
eric-haibin-lin commented on issue #2317: (info.type) != (kNotInitialized) URL: https://github.com/apache/incubator-mxnet/issues/2317#issuecomment-378094223 #8055 fixes the example test case @sxjscience provides. I'm gonna close it for now. Feel free to reopen it This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin closed issue #2317: (info.type) != (kNotInitialized)
eric-haibin-lin closed issue #2317: (info.type) != (kNotInitialized) URL: https://github.com/apache/incubator-mxnet/issues/2317 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nju-luke commented on issue #10368: asscalar is very slow
nju-luke commented on issue #10368: asscalar is very slow URL: https://github.com/apache/incubator-mxnet/issues/10368#issuecomment-378094033 @reminisce Thanks for the explanation about `asscalar`. I'm sorry that I didn't make it clear about OOM. When I said iteration, it means the epoch. That is, the OOM error is raised during the second epoch, in half the task is completed. ``` python train_softmax.py 100%|██| 300/300 [00:05<00:00, 59.88it/s] Epoch 0. Train loss: 4.608121, Valid loss 4.608134 42%|▌ | 1088/2565 [00:21<00:29, 50.49it/s]terminate called after throwing an instance of 'dmlc::Error' what(): [19:33:44] src/engine/./threaded_engine.h:359: [19:33:44] src/storage/./pooled_storage_manager.h:107: cudaMalloc failed: out of memory Stack trace returned 10 entries: ... ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178687832
##
File path: include/mxnet/ndarray.h
##
@@ -507,6 +507,35 @@ class NDArray {
ret.reuse_ = true;
return ret;
}
+
+ inline void SparseUpdateChunk(const NDArray &arr) const {
Review comment:
We definitely need to add doc for this fucntion to prevent others from
misusage
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178686915
##
File path: src/operator/custom/custom.cc
##
@@ -266,97 +267,237 @@ OpStatePtr CreateState(const NodeAttrs& attrs, Context
ctx,
return OpStatePtr::Create(state);
}
-void Forward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Forward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
std::vector ptrs;
std::vector tags;
std::vector cpys;
+ std::unordered_set input_tags({0, 4});
+ std::unordered_set output_tags({1});
auto dev_id = ctx.run_ctx.ctx.dev_id;
for (size_t i = 0; i < params.num_args; ++i) {
-NDArray *nd = new NDArray(inputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(0);
}
for (size_t i = 0; i < params.num_outs; ++i) {
-NDArray *nd = new NDArray(outputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, outputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(1);
}
for (size_t i = 0; i < params.num_auxs; ++i) {
-NDArray *nd = new NDArray(inputs[i+params.num_args], dev_id);
+size_t idx = i + params.num_args;
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, idx, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(4);
}
CustomOperator::Get()->Push(
-[=]() {
-
CHECK(reinterpret_cast(params.info->callbacks[kCustomOpForward])(
-ptrs.size(), const_cast(ptrs.data()),
const_cast(tags.data()),
-reinterpret_cast(req.data()),
static_cast(ctx.is_train),
-params.info->contexts[kCustomOpForward]));
-}, ctx, false, ctx.is_train, cpys);
+ [=]() {
+CHECK(reinterpret_cast(
+params.info->callbacks[kCustomOpForward])(
+ptrs.size(), const_cast(ptrs.data()),
+const_cast(tags.data()),
+reinterpret_cast(req.data()),
+static_cast(ctx.is_train),
+params.info->contexts[kCustomOpForward]));
+ },
+ ctx, false, ctx.is_train, cpys, tags, input_tags, output_tags, inputs,
outputs);
}
-
-void Backward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Backward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
- size_t total = 2*params.num_args + 2*params.num_outs + params.num_auxs;
- std::vector ptrs(params.num_args + 2*params.num_outs, nullptr);
+ size_t total = 2 * params.num_args + 2 * params.num_outs + params.num_auxs;
+ std::vector ptrs(params.num_args + 2 * params.num_outs, nullptr);
+
std::vector tags;
std::vector cpys;
ptrs.reserve(total);
tags.reserve(total);
+ cpys.reserve(total);
+
+ std::unordered_set input_tags({3, 0, 1, 4});
+ std::unordered_set output_tags({2});
+
for (size_t i = 0; i < params.num_outs; ++i) tags.push_back(3);
for (size_t i = 0; i < params.num_args; ++i) tags.push_back(0);
for (size_t i = 0; i < params.num_outs; ++i) tags.push_back(1);
auto dev_id = ctx.run_ctx.ctx.dev_id;
+
for (size_t i = 0; i < params.bwd_idx.size(); ++i) {
-NDArray *nd = new NDArray(inputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, i, dev_id);
cpys.push_back(*nd);
ptrs[params.bwd_idx[i]] = reinterpret_cast(nd);
}
for (size_t i = 0; i < ptrs.size(); ++i) {
-if (ptrs[i] == nullptr) ptrs[i] = reinterpret_cast(new NDArray());
+NDArray* nd;
+if (ptrs[i] == nullptr) {
+nd = new NDArray();
+ptrs[i] = reinterpret_cast(nd);
+}
}
- for (const auto& i : outputs) {
-NDArray* nd = new NDArray(i, dev_id);
+ for (size_t i = 0; i < outputs.size(); ++i) {
+NDArray* nd;
+allocate_ndarray_copy(&nd, outputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(2);
}
+
for (size_t i = 0; i < params.num_auxs; ++i) {
-NDArray* nd = new NDArray(inputs[inputs.size()-params.num_auxs+i], dev_id);
+size_t idx = inputs.size() - params.num_auxs + i;
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, idx, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(4);
}
-
CustomOperator::Get()->Push(
[=]() {
CHECK(reinte
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178687532 ## File path: tests/python/unittest/test_operator.py ## @@ -4059,6 +4059,79 @@ def create_operator(self, ctx, shapes, dtypes): with mx.contrib.autograd.train_section(): y = mx.nd.Custom(x, aux, op_type='sqr') y.backward() +y.wait_to_read() +x.grad.wait_to_read() + Review comment: Is the test case for sparse input not added here? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178687478 ## File path: tests/python/unittest/test_operator.py ## @@ -4059,6 +4059,79 @@ def create_operator(self, ctx, shapes, dtypes): with mx.contrib.autograd.train_section(): y = mx.nd.Custom(x, aux, op_type='sqr') y.backward() +y.wait_to_read() +x.grad.wait_to_read() + +# test for backward compatibility, i.e. the correctness of default implementation of +# infer storage in custom operator +class Mult(mx.operator.CustomOp): +def forward(self, is_train, req, in_data, out_data, aux): Review comment: nit: 8 space indentation? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178686409 ## File path: src/operator/custom/custom.cc ## @@ -266,97 +267,237 @@ OpStatePtr CreateState(const NodeAttrs& attrs, Context ctx, return OpStatePtr::Create(state); } -void Forward(const OpStatePtr& state, - const OpContext& ctx, - const std::vector& inputs, +void Forward(const OpStatePtr& state, const OpContext& ctx, Review comment: Maybe rename to `ForwardEx` to follow the convention for ComputeEx? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178687188
##
File path: src/operator/custom/custom.cc
##
@@ -266,97 +267,237 @@ OpStatePtr CreateState(const NodeAttrs& attrs, Context
ctx,
return OpStatePtr::Create(state);
}
-void Forward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Forward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
std::vector ptrs;
std::vector tags;
std::vector cpys;
+ std::unordered_set input_tags({0, 4});
+ std::unordered_set output_tags({1});
auto dev_id = ctx.run_ctx.ctx.dev_id;
for (size_t i = 0; i < params.num_args; ++i) {
-NDArray *nd = new NDArray(inputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(0);
}
for (size_t i = 0; i < params.num_outs; ++i) {
-NDArray *nd = new NDArray(outputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, outputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(1);
}
for (size_t i = 0; i < params.num_auxs; ++i) {
-NDArray *nd = new NDArray(inputs[i+params.num_args], dev_id);
+size_t idx = i + params.num_args;
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, idx, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(4);
}
CustomOperator::Get()->Push(
-[=]() {
-
CHECK(reinterpret_cast(params.info->callbacks[kCustomOpForward])(
-ptrs.size(), const_cast(ptrs.data()),
const_cast(tags.data()),
-reinterpret_cast(req.data()),
static_cast(ctx.is_train),
-params.info->contexts[kCustomOpForward]));
-}, ctx, false, ctx.is_train, cpys);
+ [=]() {
+CHECK(reinterpret_cast(
+params.info->callbacks[kCustomOpForward])(
+ptrs.size(), const_cast(ptrs.data()),
+const_cast(tags.data()),
+reinterpret_cast(req.data()),
+static_cast(ctx.is_train),
+params.info->contexts[kCustomOpForward]));
+ },
+ ctx, false, ctx.is_train, cpys, tags, input_tags, output_tags, inputs,
outputs);
}
-
-void Backward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Backward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
- size_t total = 2*params.num_args + 2*params.num_outs + params.num_auxs;
- std::vector ptrs(params.num_args + 2*params.num_outs, nullptr);
+ size_t total = 2 * params.num_args + 2 * params.num_outs + params.num_auxs;
+ std::vector ptrs(params.num_args + 2 * params.num_outs, nullptr);
+
std::vector tags;
std::vector cpys;
ptrs.reserve(total);
tags.reserve(total);
+ cpys.reserve(total);
+
+ std::unordered_set input_tags({3, 0, 1, 4});
+ std::unordered_set output_tags({2});
+
for (size_t i = 0; i < params.num_outs; ++i) tags.push_back(3);
for (size_t i = 0; i < params.num_args; ++i) tags.push_back(0);
for (size_t i = 0; i < params.num_outs; ++i) tags.push_back(1);
auto dev_id = ctx.run_ctx.ctx.dev_id;
+
for (size_t i = 0; i < params.bwd_idx.size(); ++i) {
-NDArray *nd = new NDArray(inputs[i], dev_id);
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, i, dev_id);
cpys.push_back(*nd);
ptrs[params.bwd_idx[i]] = reinterpret_cast(nd);
}
for (size_t i = 0; i < ptrs.size(); ++i) {
-if (ptrs[i] == nullptr) ptrs[i] = reinterpret_cast(new NDArray());
+NDArray* nd;
+if (ptrs[i] == nullptr) {
+nd = new NDArray();
+ptrs[i] = reinterpret_cast(nd);
+}
}
- for (const auto& i : outputs) {
-NDArray* nd = new NDArray(i, dev_id);
+ for (size_t i = 0; i < outputs.size(); ++i) {
+NDArray* nd;
+allocate_ndarray_copy(&nd, outputs, i, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(2);
}
+
for (size_t i = 0; i < params.num_auxs; ++i) {
-NDArray* nd = new NDArray(inputs[inputs.size()-params.num_auxs+i], dev_id);
+size_t idx = inputs.size() - params.num_auxs + i;
+NDArray* nd;
+allocate_ndarray_copy(&nd, inputs, idx, dev_id);
cpys.push_back(*nd);
ptrs.push_back(reinterpret_cast(nd));
tags.push_back(4);
}
-
CustomOperator::Get()->Push(
[=]() {
CHECK(reinte
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178687755
##
File path: include/mxnet/ndarray.h
##
@@ -507,6 +507,35 @@ class NDArray {
ret.reuse_ = true;
return ret;
}
+
+ inline void SparseUpdateChunk(const NDArray &arr) const {
+auto stype = arr.storage_type();
+CHECK(stype == kCSRStorage || stype == kRowSparseStorage)
+<< "Only to be used with CSR and RSP storage types";
+ptr_->shandle.dptr = arr.ptr_->shandle.dptr;
Review comment:
Would `ptr_->shandle = arr.ptr_->shandle ` be sufficient?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178669126 ## File path: example/numpy-ops/custom_sparse_sqr.py ## @@ -0,0 +1,115 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# pylint: skip-file +import mxnet as mx +import numpy as np + +class Sqr(mx.operator.CustomOp): +'''Example of how to use custom op with sparse ndarrays +''' +def forward(self, is_train, req, in_data, out_data, aux): +#self.assign(out_data[0], req[0], mx.nd.sparse.square(in_data[0])) Review comment: Would it make more sense to have an example without using `mx.nd.square`? For example: ``` input = in_data[0] data = input.data output = sparse.csr_matrix((data*data, input.indptr, input.indices), shape=...) self.assign(out_data[0], req[0], output) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178687355
##
File path: src/operator/custom/custom-inl.h
##
@@ -64,31 +64,59 @@ class CustomOperator {
return nullptr;
}
- template
- void Push(const Func& func,
-const OpContext& ctx,
-bool recording,
-bool training,
-const std::vector& arrs) {
+ template
+ void Push(const Func& func, const OpContext& ctx, bool recording,
Review comment:
Add some doc explaining why we need to pass inputs/outputs array?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178686280
##
File path: src/operator/operator_common.h
##
@@ -314,6 +314,32 @@ inline bool dispatch_mode_assign(DispatchMode *y, const
DispatchMode& x) {
}
#endif
+/*! \brief allocate ndarrays from existing ndarrays
Review comment:
Not sure if this is the best place to put this function. Would moving this
inside custom.h work? If we put it here it's very likely that somebody misuses
the fucntion looking at the current doc
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10374: Sparse support for Custom Op
eric-haibin-lin commented on a change in pull request #10374: Sparse support
for Custom Op
URL: https://github.com/apache/incubator-mxnet/pull/10374#discussion_r178686494
##
File path: src/operator/custom/custom.cc
##
@@ -266,97 +267,237 @@ OpStatePtr CreateState(const NodeAttrs& attrs, Context
ctx,
return OpStatePtr::Create(state);
}
-void Forward(const OpStatePtr& state,
- const OpContext& ctx,
- const std::vector& inputs,
+void Forward(const OpStatePtr& state, const OpContext& ctx,
+ const std::vector& inputs,
const std::vector& req,
- const std::vector& outputs) {
+ const std::vector& outputs) {
const CustomParam& params = state.get_state();
std::vector ptrs;
std::vector tags;
std::vector cpys;
+ std::unordered_set input_tags({0, 4});
Review comment:
Need better documentation to explain what the magic numbers are for...
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] pengzhao-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test
pengzhao-intel commented on issue #10365: [MXNET-261]Update MKLDNN & Add CPP Test URL: https://github.com/apache/incubator-mxnet/pull/10365#issuecomment-378090765 @zheng-da will double check. The tests in python2/3 MKLDNN-CPU passed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham commented on a change in pull request #10013: [MXNET-48] update on setting up Scala with MXNet and the IntelliJ IDE
aaronmarkham commented on a change in pull request #10013: [MXNET-48] update on setting up Scala with MXNet and the IntelliJ IDE URL: https://github.com/apache/incubator-mxnet/pull/10013#discussion_r178686519 ## File path: docs/tutorials/scala/mxnet_scala_on_intellij.md ## @@ -1,62 +1,295 @@ # Run MXNet Scala Examples Using the IntelliJ IDE + + This tutorial guides you through setting up a Scala project in the IntelliJ IDE and shows how to use an MXNet package from your application. ## Prerequisites: -To use this tutorial, you need: +To use this tutorial you need the following items, however after this list, installation info for macOS is provided for your benefit: + +- [Java 8 JDK](http://www.oracle.com/technetwork/java/javase/downloads/index.html) +- [Maven](https://maven.apache.org/install.html) +- [Scala](https://www.scala-lang.org/download/) - comes with IntelliJ, so you don't need to install it separately +- [MXNet Shared Library and Scala Package](#build-the-mxnet-shared-library-and-scala-package) +- [IntelliJ IDE](https://www.jetbrains.com/idea/) Review comment: Fixed the conflict... Well this is purpose of the tutorial... to setup intellij. The scala install page won't have this requirement... just an option. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: [MXNET-72] Improve sparse sgd on GPU (#10293)
This is an automated email from the ASF dual-hosted git repository.
haibin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 5245ef6 [MXNET-72] Improve sparse sgd on GPU (#10293)
5245ef6 is described below
commit 5245ef68191a6d47594bf331ec6e20ba6e93ad4c
Author: Haibin Lin
AuthorDate: Mon Apr 2 17:23:05 2018 -0700
[MXNET-72] Improve sparse sgd on GPU (#10293)
* gpu kernels
* update warning msg
---
src/operator/optimizer_op-inl.h | 137 +++-
src/operator/optimizer_op.cc| 34 +-
src/operator/optimizer_op.cu| 32 +-
3 files changed, 156 insertions(+), 47 deletions(-)
diff --git a/src/operator/optimizer_op-inl.h b/src/operator/optimizer_op-inl.h
index c611a2b..3b6bd57 100644
--- a/src/operator/optimizer_op-inl.h
+++ b/src/operator/optimizer_op-inl.h
@@ -108,8 +108,39 @@ inline void SGDUpdate(const nnvm::NodeAttrs& attrs,
/*! \brief kernel for sparse sgd
*/
+template
+struct SGDDnsRspKernel;
+
template
-struct SGDDnsRspKernel {
+struct SGDDnsRspKernel {
+ // DType is the output data type
+ // IType is row sparse idx type
+ // i is the ith element in row sparse gradient
+ template
+ MSHADOW_XINLINE static void Map(int i, const index_t row_length, DType* out,
const DType* weight,
+ const IType* grad_idx, const DType *grad_val,
+ const DType clip_gradient, const DType lr,
+ const DType wd, const DType rescale_grad) {
+using nnvm::dim_t;
+using namespace mshadow_op;
+const dim_t row_id = i / row_length;
+const dim_t col_id = i % row_length;
+const dim_t row_offset = grad_idx[row_id] * row_length;
+const dim_t data_i = row_offset + col_id;
+if (clip_gradient >= 0.0f) {
+ KERNEL_ASSIGN(out[data_i], req, (1.f - lr * wd) * weight[data_i] -
+ (lr) * mshadow_op::clip::Map(rescale_grad * grad_val[i],
clip_gradient));
+} else {
+ KERNEL_ASSIGN(out[data_i], req, (1.f - lr * wd) * weight[data_i] -
+(lr * rescale_grad) * grad_val[i]);
+}
+ }
+};
+
+/*! \brief kernel for sparse sgd
+ */
+template
+struct SGDDnsRspKernel {
// DType is the output data type
// IType is row sparse idx type
// i is the ith row in row sparse gradient
@@ -154,11 +185,15 @@ inline void SGDUpdateDnsRspImpl(const SGDParam& param,
MSHADOW_IDX_TYPE_SWITCH(grad.aux_type(rowsparse::kIdx), IType, {
MXNET_ASSIGN_REQ_SWITCH(req, req_type, {
DType* weight_data = weight.dptr();
-IType* grad_idx = grad.aux_data(rowsparse::kIdx).dptr();
-DType* grad_val = grad.data().dptr();
-index_t num_rows = grad.aux_shape(rowsparse::kIdx)[0];
-auto row_length = weight.shape_.ProdShape(1, weight.ndim());
-Kernel, xpu>::Launch(s, num_rows, row_length,
+const IType* grad_idx = grad.aux_data(rowsparse::kIdx).dptr();
+const DType* grad_val = grad.data().dptr();
+const nnvm::dim_t num_rows = grad.aux_shape(rowsparse::kIdx)[0];
+const auto row_length = weight.shape_.ProdShape(1, weight.ndim());
+size_t num_threads = num_rows;
+if (std::is_same::value) {
+ num_threads = num_rows * row_length;
+}
+Kernel, xpu>::Launch(s, num_threads,
row_length,
out->dptr(), weight_data, grad_idx, grad_val,
static_cast(param.clip_gradient),
static_cast(param.lr), static_cast(param.wd),
@@ -375,8 +410,11 @@ inline void MP_SGDMomUpdate(const nnvm::NodeAttrs& attrs,
});
}
+template
+struct SGDMomDnsRspDnsKernel;
+
template
-struct SGDMomDnsRspDnsKernel {
+struct SGDMomDnsRspDnsKernel {
template
MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
DType* mom_data, const DType* weight_data, const IType* grad_idx,
@@ -402,6 +440,33 @@ struct SGDMomDnsRspDnsKernel {
}
};
+template
+struct SGDMomDnsRspDnsKernel {
+ template
+ MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
+DType* mom_data, const DType* weight_data, const IType* grad_idx,
+const DType* grad_data, const DType clip_gradient, const DType momentum,
+const DType lr, const DType wd, const DType rescale_grad) {
+using nnvm::dim_t;
+const DType rate = lr * wd;
+const dim_t row_id = i / row_length;
+const dim_t col_id = i % row_length;
+const dim_t data_i = grad_idx[row_id] * row_length + col_id;
+if (clip_gradient >= 0.0f) {
+ mom_data[data_i] = momentum * mom_data[data_i]
+ - rate * weight_data[data_i]
+ - lr *
+ mshadow_op::clip::Map(rescale_grad * grad_data[i],
+clip_gradient);
+} else {
+ mom_data[data_i] = momentum * mom_data[data_i]
+- rate * we
[GitHub] eric-haibin-lin closed pull request #10293: [MXNET-72] Improve sparse sgd on GPU
eric-haibin-lin closed pull request #10293: [MXNET-72] Improve sparse sgd on GPU
URL: https://github.com/apache/incubator-mxnet/pull/10293
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/src/operator/optimizer_op-inl.h b/src/operator/optimizer_op-inl.h
index 104f20a61ee..66e5059d6ea 100644
--- a/src/operator/optimizer_op-inl.h
+++ b/src/operator/optimizer_op-inl.h
@@ -108,8 +108,39 @@ inline void SGDUpdate(const nnvm::NodeAttrs& attrs,
/*! \brief kernel for sparse sgd
*/
+template
+struct SGDDnsRspKernel;
+
template
-struct SGDDnsRspKernel {
+struct SGDDnsRspKernel {
+ // DType is the output data type
+ // IType is row sparse idx type
+ // i is the ith element in row sparse gradient
+ template
+ MSHADOW_XINLINE static void Map(int i, const index_t row_length, DType* out,
const DType* weight,
+ const IType* grad_idx, const DType *grad_val,
+ const DType clip_gradient, const DType lr,
+ const DType wd, const DType rescale_grad) {
+using nnvm::dim_t;
+using namespace mshadow_op;
+const dim_t row_id = i / row_length;
+const dim_t col_id = i % row_length;
+const dim_t row_offset = grad_idx[row_id] * row_length;
+const dim_t data_i = row_offset + col_id;
+if (clip_gradient >= 0.0f) {
+ KERNEL_ASSIGN(out[data_i], req, (1.f - lr * wd) * weight[data_i] -
+ (lr) * mshadow_op::clip::Map(rescale_grad * grad_val[i],
clip_gradient));
+} else {
+ KERNEL_ASSIGN(out[data_i], req, (1.f - lr * wd) * weight[data_i] -
+(lr * rescale_grad) * grad_val[i]);
+}
+ }
+};
+
+/*! \brief kernel for sparse sgd
+ */
+template
+struct SGDDnsRspKernel {
// DType is the output data type
// IType is row sparse idx type
// i is the ith row in row sparse gradient
@@ -154,11 +185,15 @@ inline void SGDUpdateDnsRspImpl(const SGDParam& param,
MSHADOW_IDX_TYPE_SWITCH(grad.aux_type(rowsparse::kIdx), IType, {
MXNET_ASSIGN_REQ_SWITCH(req, req_type, {
DType* weight_data = weight.dptr();
-IType* grad_idx = grad.aux_data(rowsparse::kIdx).dptr();
-DType* grad_val = grad.data().dptr();
-index_t num_rows = grad.aux_shape(rowsparse::kIdx)[0];
-auto row_length = weight.shape_.ProdShape(1, weight.ndim());
-Kernel, xpu>::Launch(s, num_rows, row_length,
+const IType* grad_idx = grad.aux_data(rowsparse::kIdx).dptr();
+const DType* grad_val = grad.data().dptr();
+const nnvm::dim_t num_rows = grad.aux_shape(rowsparse::kIdx)[0];
+const auto row_length = weight.shape_.ProdShape(1, weight.ndim());
+size_t num_threads = num_rows;
+if (std::is_same::value) {
+ num_threads = num_rows * row_length;
+}
+Kernel, xpu>::Launch(s, num_threads,
row_length,
out->dptr(), weight_data, grad_idx, grad_val,
static_cast(param.clip_gradient),
static_cast(param.lr), static_cast(param.wd),
@@ -375,8 +410,11 @@ inline void MP_SGDMomUpdate(const nnvm::NodeAttrs& attrs,
});
}
+template
+struct SGDMomDnsRspDnsKernel;
+
template
-struct SGDMomDnsRspDnsKernel {
+struct SGDMomDnsRspDnsKernel {
template
MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
DType* mom_data, const DType* weight_data, const IType* grad_idx,
@@ -402,6 +440,33 @@ struct SGDMomDnsRspDnsKernel {
}
};
+template
+struct SGDMomDnsRspDnsKernel {
+ template
+ MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
+DType* mom_data, const DType* weight_data, const IType* grad_idx,
+const DType* grad_data, const DType clip_gradient, const DType momentum,
+const DType lr, const DType wd, const DType rescale_grad) {
+using nnvm::dim_t;
+const DType rate = lr * wd;
+const dim_t row_id = i / row_length;
+const dim_t col_id = i % row_length;
+const dim_t data_i = grad_idx[row_id] * row_length + col_id;
+if (clip_gradient >= 0.0f) {
+ mom_data[data_i] = momentum * mom_data[data_i]
+ - rate * weight_data[data_i]
+ - lr *
+ mshadow_op::clip::Map(rescale_grad * grad_data[i],
+clip_gradient);
+} else {
+ mom_data[data_i] = momentum * mom_data[data_i]
+- rate * weight_data[data_i]
+- lr * rescale_grad * grad_data[i];
+}
+KERNEL_ASSIGN(out_data[data_i], req, weight_data[data_i] +
mom_data[data_i]);
+ }
+};
+
template
inline void SGDMomUpdateDnsRspDnsImpl(const SGDMomParam& param,
const OpContext& ctx,
@@ -428,7 +493,11 @@ inline void SGDMomUpdateDnsRspDnsImpl(con
[GitHub] marcoabreu opened a new issue #9000: Flaky test OOM: test_optimizers:test_sgd
marcoabreu opened a new issue #9000: Flaky test OOM: test_optimizers:test_sgd
URL: https://github.com/apache/incubator-mxnet/issues/9000
tests/ci_build/ci_build.sh gpu_mklml PYTHONPATH=./python/ nosetests-3.4
--with-timer --verbose tests/python/gpu
```
test_operator_gpu.test_sgd ... terminate called after throwing an instance
of 'dmlc::Error'
what(): [01:21:52] src/engine/./threaded_engine.h:359: [01:21:52]
src/storage/./pooled_storage_manager.h:107: cudaMalloc failed: out of memory
Stack trace returned 10 entries:
[bt] (0)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc10StackTraceB5cxx11Ev+0x5b)
[0x7f3adc78f7cb]
[bt] (1)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x28)
[0x7f3adc790218]
[bt] (2)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet7storage23GPUPooledStorageManager5AllocEPNS_7Storage6HandleE+0x16d)
[0x7f3adf0eb82d]
[bt] (3)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet11StorageImpl5AllocEPNS_7Storage6HandleE+0x5d)
[0x7f3adf0ef78d]
[bt] (4)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZNK5mxnet7NDArray13CheckAndAllocEv+0x122)
[0x7f3adca7c9a2]
[bt] (5)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet6common22SetupDefaultBlobsInOutERKSt6vectorINS_7NDArrayESaIS2_EES6_PS1_INS_5TBlobESaIS7_EESA_PS4_SB_SB_SB_PSt13unordered_mapIjjSt4hashIjESt8equal_toIjESaISt4pairIKjjEEERKS1_IjSaIjEE+0x1cf8)
[0x7f3adecbe2b8]
[bt] (6)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZZN5mxnet10imperative12PushFComputeERKSt8functionIFvRKN4nnvm9NodeAttrsERKNS_9OpContextERKSt6vectorINS_5TBlobESaISA_EERKS9_INS_9OpReqTypeESaISF_EESE_EEPKNS2_2OpES5_RKNS_7ContextERKS9_IPNS_6engine3VarESaISW_EES10_RKS9_INS_8ResourceESaIS11_EERKS9_IPNS_7NDArrayESaIS17_EES1B_RKS9_IjSaIjEESJ_ENKUlNS_10RunContextEE_clES1G_+0x1be)
[0x7f3adecbf3ae]
[bt] (7) /workspace/python/mxnet/../../lib/libmxnet.so(+0x31d2e2b)
[0x7f3adf0d1e2b]
[bt] (8)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet6engine14ThreadedEngine15ExecuteOprBlockENS_10RunContextEPNS0_8OprBlockE+0x100)
[0x7f3adf0de4d0]
[bt] (9)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet6engine23ThreadedEnginePerDevice9GPUWorkerILN4dmlc19ConcurrentQueueTypeE0EEEvNS_7ContextEbPNS1_17ThreadWorkerBlockIXT_EEESt10shared_ptrINS0_10ThreadPool11SimpleEventEE+0x10b)
[0x7f3adf0e6a4b]
A fatal error occurred in asynchronous engine operation. If you do not know
what caused this error, you can try set environment variable MXNET_ENGINE_TYPE
to NaiveEngine and run with debugger (i.e. gdb). This will force all operations
to be synchronous and backtrace will give you the series of calls that lead to
this error. Remember to set MXNET_ENGINE_TYPE back to empty after debugging.
Stack trace returned 9 entries:
[bt] (0)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc10StackTraceB5cxx11Ev+0x5b)
[0x7f3adc78f7cb]
[bt] (1)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x28)
[0x7f3adc790218]
[bt] (2)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet6engine14ThreadedEngine15ExecuteOprBlockENS_10RunContextEPNS0_8OprBlockE+0x39a)
[0x7f3adf0de76a]
[bt] (3)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet6engine23ThreadedEnginePerDevice9GPUWorkerILN4dmlc19ConcurrentQueueTypeE0EEEvNS_7ContextEbPNS1_17ThreadWorkerBlockIXT_EEESt10shared_ptrINS0_10ThreadPool11SimpleEventEE+0x10b)
[0x7f3adf0e6a4b]
[bt] (4)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZNSt17_Function_handlerIFvSt10shared_ptrIN5mxnet6engine10ThreadPool11SimpleEventEEEZZNS2_23ThreadedEnginePerDevice13PushToExecuteEPNS2_8OprBlockEbENKUlvE1_clEvEUlS5_E_E9_M_invokeERKSt9_Any_dataOS5_+0x63)
[0x7f3adf0e6c63]
[bt] (5)
/workspace/python/mxnet/../../lib/libmxnet.so(_ZNSt6thread5_ImplISt12_Bind_simpleIFSt8functionIFvSt10shared_ptrIN5mxnet6engine10ThreadPool11SimpleEventS8_EEE6_M_runEv+0x4a)
[0x7f3adf0e0bca]
[bt] (6) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7f3af740ac80]
[bt] (7) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f3afed1c6ba]
[bt] (8) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3afea523dd]
tests/ci_build/with_the_same_user: line 30: 80003 Aborted
(core dumped) sudo -u "#${CI_BUILD_UID}" --preserve-env
"LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" "HOME=${CI_BUILD_HOME}" ${COMMAND[@]}
script returned exit code 134
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on issue #9000: Flaky test OOM: test_optimizers:test_sgd
eric-haibin-lin commented on issue #9000: Flaky test OOM: test_optimizers:test_sgd URL: https://github.com/apache/incubator-mxnet/issues/9000#issuecomment-378085649 Is this fixed? Looks like the test is not enabled yet This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #10285: [MXNET-241] Module API for distributed training w/ row_sparse weight
eric-haibin-lin commented on issue #10285: [MXNET-241] Module API for distributed training w/ row_sparse weight URL: https://github.com/apache/incubator-mxnet/pull/10285#issuecomment-378084842 Added more description to `update` and `prepare` to explain why we need this function and what the argument means. @piiswrong This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] szhengac opened a new issue #10377: Inconsistency between ndarray and symbol when performing division
szhengac opened a new issue #10377: Inconsistency between ndarray and symbol when performing division URL: https://github.com/apache/incubator-mxnet/issues/10377 When doing division with ndarray, we can write `x[:] /= F.sum(x, axis=-1)` But with symbol, we need to use `x = F.broadcast_div(x, F.sum(x, axis=-1, keepdims=True))` I think this inconsistency is not friendly to user when they would like to use the HybridBlock. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #10375: [MXNET-187] [WIP] fake shuffle functions
cjolivier01 commented on issue #10375: [MXNET-187] [WIP] fake shuffle functions URL: https://github.com/apache/incubator-mxnet/pull/10375#issuecomment-378082454 It may not be worth supporting Fermi architecture This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #9410: Training with the same parameters and seed gets significantly different results
cjolivier01 commented on issue #9410: Training with the same parameters and seed gets significantly different results URL: https://github.com/apache/incubator-mxnet/issues/9410#issuecomment-378081463 Is this supposed to take a really long time to run? It takes many minutes... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #10373: Adding a BSD file to LICENSE
eric-haibin-lin commented on issue #10373: Adding a BSD file to LICENSE URL: https://github.com/apache/incubator-mxnet/pull/10373#issuecomment-378078762 Thanks for fixing this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Adding a file to LICENSE (#10373)
This is an automated email from the ASF dual-hosted git repository. haibin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 6dd85e2 Adding a file to LICENSE (#10373) 6dd85e2 is described below commit 6dd85e2da9bfba13c8b3a6b384d82fccb0b607d7 Author: mbaijal <30911248+mbai...@users.noreply.github.com> AuthorDate: Mon Apr 2 16:20:14 2018 -0700 Adding a file to LICENSE (#10373) --- LICENSE | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/LICENSE b/LICENSE index a9ec986..ae18eff 100644 --- a/LICENSE +++ b/LICENSE @@ -395,7 +395,7 @@ === 4. FindCrypto.cmake -For details, see, dmlc-core/cmake/Modules/FindCrypto.cmake, +For details, see, 3rdparty/dmlc-core/cmake/Modules/FindCrypto.cmake, Redistribution and use is allowed according to the terms of the BSD license. === @@ -497,15 +497,20 @@ 8. Semaphore implementation in blockingconcurrentqueue.h This file uses a semaphore implementation under the terms of its separate zlib license. -For details, see, dmlc-core/include/dmlc/blockingconcurrentqueue.h +For details, see, 3rdparty/dmlc-core/include/dmlc/blockingconcurrentqueue.h === 9. blockingconcurrentqueue.h This file is Distributed under the terms of the simplified BSD license. -For details, see, dmlc-core/include/dmlc/blockingconcurrentqueue.h +For details, see, 3rdparty/dmlc-core/include/dmlc/blockingconcurrentqueue.h === +10. concurrentqueue.h +This file is Distributed under the terms of the simplified BSD license. +For details, see, 3rdparty/dmlc-core/include/dmlc/concurrentqueue.h + + === -- To stop receiving notification emails like this one, please contact hai...@apache.org.
[GitHub] eric-haibin-lin closed pull request #10373: Adding a BSD file to LICENSE
eric-haibin-lin closed pull request #10373: Adding a BSD file to LICENSE URL: https://github.com/apache/incubator-mxnet/pull/10373 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/LICENSE b/LICENSE index b783e3ce3ca..b79a3c548ac 100644 --- a/LICENSE +++ b/LICENSE @@ -395,7 +395,7 @@ === 4. FindCrypto.cmake -For details, see, dmlc-core/cmake/Modules/FindCrypto.cmake, +For details, see, 3rdparty/dmlc-core/cmake/Modules/FindCrypto.cmake, Redistribution and use is allowed according to the terms of the BSD license. === @@ -497,15 +497,20 @@ 8. Semaphore implementation in blockingconcurrentqueue.h This file uses a semaphore implementation under the terms of its separate zlib license. -For details, see, dmlc-core/include/dmlc/blockingconcurrentqueue.h +For details, see, 3rdparty/dmlc-core/include/dmlc/blockingconcurrentqueue.h === 9. blockingconcurrentqueue.h This file is Distributed under the terms of the simplified BSD license. -For details, see, dmlc-core/include/dmlc/blockingconcurrentqueue.h +For details, see, 3rdparty/dmlc-core/include/dmlc/blockingconcurrentqueue.h === +10. concurrentqueue.h +This file is Distributed under the terms of the simplified BSD license. +For details, see, 3rdparty/dmlc-core/include/dmlc/concurrentqueue.h + + === This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lanking520 commented on a change in pull request #10346: [MXNET-256] Add CI Test for GPU
lanking520 commented on a change in pull request #10346: [MXNET-256] Add CI
Test for GPU
URL: https://github.com/apache/incubator-mxnet/pull/10346#discussion_r178676143
##
File path: ci/docker/runtime_functions.sh
##
@@ -427,6 +427,12 @@ unittest_ubuntu_cpu_scala() {
make scalatest USE_BLAS=openblas
}
+unittest_ubuntu_gpu_scala() {
+set -ex
+make scalapkg USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1
USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1
Review comment:
Added to the test section
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10013: [MXNET-48] update on setting up Scala with MXNet and the IntelliJ IDE
eric-haibin-lin commented on a change in pull request #10013: [MXNET-48] update on setting up Scala with MXNet and the IntelliJ IDE URL: https://github.com/apache/incubator-mxnet/pull/10013#discussion_r178675676 ## File path: docs/tutorials/scala/mxnet_scala_on_intellij.md ## @@ -1,62 +1,295 @@ # Run MXNet Scala Examples Using the IntelliJ IDE + + This tutorial guides you through setting up a Scala project in the IntelliJ IDE and shows how to use an MXNet package from your application. ## Prerequisites: -To use this tutorial, you need: +To use this tutorial you need the following items, however after this list, installation info for macOS is provided for your benefit: + +- [Java 8 JDK](http://www.oracle.com/technetwork/java/javase/downloads/index.html) +- [Maven](https://maven.apache.org/install.html) +- [Scala](https://www.scala-lang.org/download/) - comes with IntelliJ, so you don't need to install it separately +- [MXNet Shared Library and Scala Package](#build-the-mxnet-shared-library-and-scala-package) +- [IntelliJ IDE](https://www.jetbrains.com/idea/) Review comment: Do we really want to have IDEs like IntelliJ as part of the prerequisite? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] szha commented on a change in pull request #10354: Expose the number of GPUs.
szha commented on a change in pull request #10354: Expose the number of GPUs.
URL: https://github.com/apache/incubator-mxnet/pull/10354#discussion_r178675441
##
File path: python/mxnet/context.py
##
@@ -212,6 +216,14 @@ def gpu(device_id=0):
return Context('gpu', device_id)
+def num_gpus():
+count = ctypes.c_int()
+check_call(_LIB.MXGetGPUCount(ctypes.byref(count)))
+return count.value
+
+def gpus():
+return [gpu(idx) for idx in range(0, num_gpus())]
Review comment:
can there ever be a case where the gpu device indices are not consecutive or
not starting from 0?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-mxnet] branch master updated: [MXNET-146] Docs build updates: added some deps; clarified developer builds (#10270)
This is an automated email from the ASF dual-hosted git repository. haibin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new c3c2676 [MXNET-146] Docs build updates: added some deps; clarified developer builds (#10270) c3c2676 is described below commit c3c26769067530f7deb1e1220068ddc97ef51fff Author: Aaron Markham AuthorDate: Mon Apr 2 16:11:40 2018 -0700 [MXNET-146] Docs build updates: added some deps; clarified developer builds (#10270) * added some deps; clarified developer builds * clarifications and notes on developer builds * copy command to include hidden files --- docs/README.md | 95 ++- docs/build_version_doc/Dockerfile | 1 + docs/build_version_doc/README.md| 136 +++- docs/build_version_doc/setup_docs_ubuntu.sh | 24 +++-- 4 files changed, 176 insertions(+), 80 deletions(-) diff --git a/docs/README.md b/docs/README.md index ad64b76..d62a73c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,91 +1,54 @@ -# MXNet documentation +# MXNet Documentation -## How to build MXNet website +The website is hosted at http://mxnet.incubator.apache.org/. +http://mxnet.io redirects to this site and advised to use links with http://mxnet.incubator.apache.org/ instead of http://mxnet.io/. -MXNet Documentation Website is built with [sphinx 1.5.1](http://www.sphinx-doc.org/en/1.5.1/intro.html). +MXNet Documentation Website is built with [Sphinx](http://www.sphinx-doc.org) and a variety of plugins including [pandoc](https://pandoc.org/), [recommonmark](https://github.com/rtfd/recommonmark), a custom Sphinx plugin ([mxdoc.py](https://github.com/apache/incubator-mxnet/blob/master/docs/mxdoc.py)). -A built version of document is available at http://mxnet.io -To build the documents locally, we need to first install [docker](https://docker.com). -Then use the following commands to clone and -build the documents. +## How to Build the MXNet Website for Development and QA -```bash -git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet -cd mxnet && make docs -``` +* [Dependencies](build_doc_version/README.md#dependencies) +* [Developer Build Instructions](build_doc_version/README.md#developer-instructions) +* [Full Site Build Instructions](build_doc_version/README.md#full-website-build) -In case docker method is not available, there is an alternate method: -```bash -sudo pip install sphinx==1.5.1 CommonMark==0.5.4 breathe mock==1.0.1 recommonmark pypandoc -cd mxnet/docs && make html USE_OPENMP=0 -``` -The results will be available at `docs/_build/html/`. +## File Structure -Note: +* Static files such as **css**, **javascript** and **html** templates are under the `_static` folder: + - Javascript files are under `_static/js` folder + - Layout templates and landing page html file are under `_static/mxnet-theme` folder + - `_static/mxnet.css` contains all MXNet website styles -- If C++ codes have been changed, we suggest to remove the previous results to - trigger the rebuild for all pages, namely run `make clean_docs`. -- If C++ code fails to build, run `make clean` -- If CSS or javascript are changed, we often need to do a *force refresh* in the - browser to clear the cache. -- If search doesn't work, we need to `make clean` and rebuild. - -## File structure +* Page contents originate as markdown files. Sphinx converts markdown files to html through an `rst` intermediate format. Each content folder should contain an index file as landing page. -1. Static files such as css, javascript and html templates are under `_static` folder: -- Javascript files are under `_static/js` folder. -- Layout templates and landing page html file are under `_static/mxnet-theme` folder. -- `_static/mxnet.css` contains all MXNet website styles. +* There are some utility scripts to help building website, such as `mxdoc.py` and `build_version_doc/`. They are used to manipulate website contents during building. Refer to [Developer Build Instructions](build_doc_version/README.md#developer-instructions) for more information. -2. Sphinx converts markdowns files to html. Page contents are markdown files. Each content folder -contains an index file as landing page. -3. There are some utility scripts to help building website, such as `mxdoc.py` and `build_version_doc/`. -They are used to manipulate website contents during building. +## Production Website Building Process -## Production website building process +**IMPORTANT**: this is currently offline. -[Apache Jenkins MXNet website building job](https://builds.apache.org/job/incubator-mxnet-build-site/) is used to build MXNet website. -There are two ways to trigger this job. -First is nightly build for master branch. +[Apache Jenkins MXNet website building
[GitHub] eric-haibin-lin closed pull request #10270: [MXNET-146] Docs build updates: added some deps; clarified developer builds
eric-haibin-lin closed pull request #10270: [MXNET-146] Docs build updates: added some deps; clarified developer builds URL: https://github.com/apache/incubator-mxnet/pull/10270 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/README.md b/docs/README.md index ad64b76d648..d62a73c9621 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,91 +1,54 @@ -# MXNet documentation +# MXNet Documentation -## How to build MXNet website +The website is hosted at http://mxnet.incubator.apache.org/. +http://mxnet.io redirects to this site and advised to use links with http://mxnet.incubator.apache.org/ instead of http://mxnet.io/. -MXNet Documentation Website is built with [sphinx 1.5.1](http://www.sphinx-doc.org/en/1.5.1/intro.html). +MXNet Documentation Website is built with [Sphinx](http://www.sphinx-doc.org) and a variety of plugins including [pandoc](https://pandoc.org/), [recommonmark](https://github.com/rtfd/recommonmark), a custom Sphinx plugin ([mxdoc.py](https://github.com/apache/incubator-mxnet/blob/master/docs/mxdoc.py)). -A built version of document is available at http://mxnet.io -To build the documents locally, we need to first install [docker](https://docker.com). -Then use the following commands to clone and -build the documents. +## How to Build the MXNet Website for Development and QA -```bash -git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet -cd mxnet && make docs -``` +* [Dependencies](build_doc_version/README.md#dependencies) +* [Developer Build Instructions](build_doc_version/README.md#developer-instructions) +* [Full Site Build Instructions](build_doc_version/README.md#full-website-build) -In case docker method is not available, there is an alternate method: -```bash -sudo pip install sphinx==1.5.1 CommonMark==0.5.4 breathe mock==1.0.1 recommonmark pypandoc -cd mxnet/docs && make html USE_OPENMP=0 -``` -The results will be available at `docs/_build/html/`. +## File Structure -Note: +* Static files such as **css**, **javascript** and **html** templates are under the `_static` folder: + - Javascript files are under `_static/js` folder + - Layout templates and landing page html file are under `_static/mxnet-theme` folder + - `_static/mxnet.css` contains all MXNet website styles -- If C++ codes have been changed, we suggest to remove the previous results to - trigger the rebuild for all pages, namely run `make clean_docs`. -- If C++ code fails to build, run `make clean` -- If CSS or javascript are changed, we often need to do a *force refresh* in the - browser to clear the cache. -- If search doesn't work, we need to `make clean` and rebuild. - -## File structure +* Page contents originate as markdown files. Sphinx converts markdown files to html through an `rst` intermediate format. Each content folder should contain an index file as landing page. -1. Static files such as css, javascript and html templates are under `_static` folder: -- Javascript files are under `_static/js` folder. -- Layout templates and landing page html file are under `_static/mxnet-theme` folder. -- `_static/mxnet.css` contains all MXNet website styles. +* There are some utility scripts to help building website, such as `mxdoc.py` and `build_version_doc/`. They are used to manipulate website contents during building. Refer to [Developer Build Instructions](build_doc_version/README.md#developer-instructions) for more information. -2. Sphinx converts markdowns files to html. Page contents are markdown files. Each content folder -contains an index file as landing page. -3. There are some utility scripts to help building website, such as `mxdoc.py` and `build_version_doc/`. -They are used to manipulate website contents during building. +## Production Website Building Process -## Production website building process +**IMPORTANT**: this is currently offline. -[Apache Jenkins MXNet website building job](https://builds.apache.org/job/incubator-mxnet-build-site/) is used to build MXNet website. -There are two ways to trigger this job. -First is nightly build for master branch. +[Apache Jenkins MXNet website building job](https://builds.apache.org/job/incubator-mxnet-build-site/) is used to build MXNet website. +There are two ways to trigger this job. +First is nightly build for master branch. Second is manually trigger job when a new version is released. This will build for new version. -The job will fetch mxnet repository, build MXNet website and push all static files to [host repository](https://github.com/apache/incubator-mxnet-site.git). +The job will fetch mxnet repository, build MXNet website and push all static files to [host repository](https://github.com/apache/incubator-mxnet-site.git).
[GitHub] szha commented on issue #10360: extend ndarray in-place reshape
szha commented on issue #10360: extend ndarray in-place reshape URL: https://github.com/apache/incubator-mxnet/pull/10360#issuecomment-378076559 Doc can be found at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-10360/6/api/python/ndarray/ndarray.html#mxnet.ndarray.NDArray.reshape This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] haojin2 commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
haojin2 commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178673220 ## File path: src/operator/tensor/elemwise_binary_broadcast_op_basic.cc ## @@ -120,8 +120,13 @@ Example:: broadcast_mul(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]] Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] indhub closed pull request #10283: [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon
indhub closed pull request #10283: [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon URL: https://github.com/apache/incubator-mxnet/pull/10283 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/api/python/contrib/onnx.md b/docs/api/python/contrib/onnx.md index 2a8f1fcea0a..44aabaf4419 100644 --- a/docs/api/python/contrib/onnx.md +++ b/docs/api/python/contrib/onnx.md @@ -33,6 +33,7 @@ This document describes all the ONNX-MXNet APIs. /tutorials/onnx/super_resolution.md /tutorials/onnx/inference_on_onnx_model.md + /tutorials/onnx/fine_tuning_gluon.md ``` ## API Reference diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 62cb5ebcdee..4aba4a68a0f 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -111,20 +111,21 @@ The Gluon and Module tutorials are in Python, but you can also find a variety of -- [Creating custom operators with numpy](http://mxnet.incubator.apache.org/tutorials/gluon/customop.html) +- [Creating custom operators with numpy](/tutorials/gluon/customop.html) -- [Handwritten digit recognition (MNIST)](http://mxnet.incubator.apache.org/tutorials/gluon/mnist.html) +- [Handwritten digit recognition (MNIST)](/tutorials/gluon/mnist.html) -- [Hybrid network example](http://mxnet.incubator.apache.org/tutorials/gluon/hybrid.html) +- [Hybrid network example](/tutorials/gluon/hybrid.html) -- [Neural network building blocks with gluon](http://mxnet.incubator.apache.org/tutorials/gluon/gluon.html) +- [Neural network building blocks with gluon](/tutorials/gluon/gluon.html) -- [Simple autograd example](http://mxnet.incubator.apache.org/tutorials/gluon/autograd.html) +- [Simple autograd example](/tutorials/gluon/autograd.html) -- [Data Augmentation with Masks (for Object Segmentation)](http://mxnet.incubator.apache.org/tutorials/python/data_augmentation_with_masks.html) +- [Data Augmentation with Masks (for Object Segmentation)](/tutorials/python/data_augmentation_with_masks.html) -- [Inference using an ONNX model](http://mxnet.incubator.apache.org/tutorials/onnx/inference_on_onnx_model.html) +- [Inference using an ONNX model](/tutorials/onnx/inference_on_onnx_model.html) +- [Fine-tuning an ONNX model on Gluon](/tutorials/onnx/fine_tuning_gluon.html) @@ -138,42 +139,42 @@ The Gluon and Module tutorials are in Python, but you can also find a variety of -- [Imperative tensor operations on CPU/GPU](http://mxnet.incubator.apache.org/tutorials/basic/ndarray.html) +- [Imperative tensor operations on CPU/GPU](/tutorials/basic/ndarray.html) -- [NDArray Indexing](../tutorials/basic/ndarray_indexing.html) +- [NDArray Indexing](/tutorials/basic/ndarray_indexing.html) -- [Symbol API](http://mxnet.incubator.apache.org/tutorials/basic/symbol.html) +- [Symbol API](/tutorials/basic/symbol.html) -- [Module API](http://mxnet.incubator.apache.org/tutorials/basic/module.html) +- [Module API](/tutorials/basic/module.html) -- [Iterators - Loading data](http://mxnet.incubator.apache.org/tutorials/basic/data.html) +- [Iterators - Loading data](/tutorials/basic/data.html) -- [Linear regression](http://mxnet.incubator.apache.org/tutorials/python/linear-regression.html) +- [Linear regression](/tutorials/python/linear-regression.html) -- [MNIST - handwriting recognition](http://mxnet.incubator.apache.org/tutorials/python/mnist.html) +- [MNIST - handwriting recognition](/tutorials/python/mnist.html) -- [Large scale image classification](http://mxnet.incubator.apache.org/tutorials/vision/large_scale_classification.html) +- [Large scale image classification](/tutorials/vision/large_scale_classification.html) -- [NDArray in Compressed Sparse Row storage format](http://mxnet.incubator.apache.org/tutorials/sparse/csr.html) +- [NDArray in Compressed Sparse Row storage format](/tutorials/sparse/csr.html) -- [Sparse gradient updates](http://mxnet.incubator.apache.org/tutorials/sparse/row_sparse.html) +- [Sparse gradient updates](/tutorials/sparse/row_sparse.html) -- [Train a linear regression model with sparse symbols](http://mxnet.incubator.apache.org/tutorials/sparse/train.html) +- [Train a linear regression model with sparse symbols](/tutorials/sparse/train.html) -- [Types of Data Augmentation](http://mxnet.incubator.apache.org/tutorials/python/types_of_data_augmentation.html) +- [Types of data augmentation](/tutorials/python/types_of_data_augmentation.html) @@ -184,15 +185,15 @@ The Gluon and Module tutorials are in Python, but you can also find a variety of - [Connectionist Temporal Classification](../tutorials/speech_recognition/ctc.html) -- [Distributed key-value store](http://mxnet.incubator.apache.org/tutorials/python/kvstore.ht
[incubator-mxnet] branch master updated: [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon (#10283)
This is an automated email from the ASF dual-hosted git repository. indhub pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 224b0c9 [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon (#10283) 224b0c9 is described below commit 224b0c9fc9a9bc733e70199509dc177d62f71aaa Author: ThomasDelteil AuthorDate: Mon Apr 2 15:24:55 2018 -0700 [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon (#10283) * ONNX fine-tuning tutorial * Update inference onnx notebook with feedback * Update inference_on_onnx_model.md * Update fine_tuning_gluon.md * adding the onnx tutorial --- docs/api/python/contrib/onnx.md| 1 + docs/tutorials/index.md| 77 ++--- docs/tutorials/onnx/fine_tuning_gluon.md | 433 + docs/tutorials/onnx/inference_on_onnx_model.md | 53 ++- 4 files changed, 492 insertions(+), 72 deletions(-) diff --git a/docs/api/python/contrib/onnx.md b/docs/api/python/contrib/onnx.md index 2a8f1fc..44aabaf 100644 --- a/docs/api/python/contrib/onnx.md +++ b/docs/api/python/contrib/onnx.md @@ -33,6 +33,7 @@ This document describes all the ONNX-MXNet APIs. /tutorials/onnx/super_resolution.md /tutorials/onnx/inference_on_onnx_model.md + /tutorials/onnx/fine_tuning_gluon.md ``` ## API Reference diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 62cb5eb..4aba4a6 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -111,20 +111,21 @@ The Gluon and Module tutorials are in Python, but you can also find a variety of -- [Creating custom operators with numpy](http://mxnet.incubator.apache.org/tutorials/gluon/customop.html) +- [Creating custom operators with numpy](/tutorials/gluon/customop.html) -- [Handwritten digit recognition (MNIST)](http://mxnet.incubator.apache.org/tutorials/gluon/mnist.html) +- [Handwritten digit recognition (MNIST)](/tutorials/gluon/mnist.html) -- [Hybrid network example](http://mxnet.incubator.apache.org/tutorials/gluon/hybrid.html) +- [Hybrid network example](/tutorials/gluon/hybrid.html) -- [Neural network building blocks with gluon](http://mxnet.incubator.apache.org/tutorials/gluon/gluon.html) +- [Neural network building blocks with gluon](/tutorials/gluon/gluon.html) -- [Simple autograd example](http://mxnet.incubator.apache.org/tutorials/gluon/autograd.html) +- [Simple autograd example](/tutorials/gluon/autograd.html) -- [Data Augmentation with Masks (for Object Segmentation)](http://mxnet.incubator.apache.org/tutorials/python/data_augmentation_with_masks.html) +- [Data Augmentation with Masks (for Object Segmentation)](/tutorials/python/data_augmentation_with_masks.html) -- [Inference using an ONNX model](http://mxnet.incubator.apache.org/tutorials/onnx/inference_on_onnx_model.html) +- [Inference using an ONNX model](/tutorials/onnx/inference_on_onnx_model.html) +- [Fine-tuning an ONNX model on Gluon](/tutorials/onnx/fine_tuning_gluon.html) @@ -138,42 +139,42 @@ The Gluon and Module tutorials are in Python, but you can also find a variety of -- [Imperative tensor operations on CPU/GPU](http://mxnet.incubator.apache.org/tutorials/basic/ndarray.html) +- [Imperative tensor operations on CPU/GPU](/tutorials/basic/ndarray.html) -- [NDArray Indexing](../tutorials/basic/ndarray_indexing.html) +- [NDArray Indexing](/tutorials/basic/ndarray_indexing.html) -- [Symbol API](http://mxnet.incubator.apache.org/tutorials/basic/symbol.html) +- [Symbol API](/tutorials/basic/symbol.html) -- [Module API](http://mxnet.incubator.apache.org/tutorials/basic/module.html) +- [Module API](/tutorials/basic/module.html) -- [Iterators - Loading data](http://mxnet.incubator.apache.org/tutorials/basic/data.html) +- [Iterators - Loading data](/tutorials/basic/data.html) -- [Linear regression](http://mxnet.incubator.apache.org/tutorials/python/linear-regression.html) +- [Linear regression](/tutorials/python/linear-regression.html) -- [MNIST - handwriting recognition](http://mxnet.incubator.apache.org/tutorials/python/mnist.html) +- [MNIST - handwriting recognition](/tutorials/python/mnist.html) -- [Large scale image classification](http://mxnet.incubator.apache.org/tutorials/vision/large_scale_classification.html) +- [Large scale image classification](/tutorials/vision/large_scale_classification.html) -- [NDArray in Compressed Sparse Row storage format](http://mxnet.incubator.apache.org/tutorials/sparse/csr.html) +- [NDArray in Compressed Sparse Row storage format](/tutorials/sparse/csr.html) -- [Sparse gradient updates](http://mxnet.incubator.apache.org/tutorials/sparse/row_sparse.html) +- [Sparse gradient updates](/tutorials/sparse/row_sparse.html) -- [Train a linear regression model with sparse symbols](http://mxnet.incubator.a
[GitHub] eric-haibin-lin commented on a change in pull request #10183: [MXNET-120] Float16 support for distributed training
eric-haibin-lin commented on a change in pull request #10183: [MXNET-120]
Float16 support for distributed training
URL: https://github.com/apache/incubator-mxnet/pull/10183#discussion_r178665723
##
File path: src/kvstore/kvstore_dist_server.h
##
@@ -170,43 +216,90 @@ class KVStoreDistServer {
app->Response(recved);
}
+ /*
+ * For keys already initialized, if necessary create stored_realt.
+ * This will only be used if by some wrong usage of kvstore,
+ * some keys are initialized before optimizer is set.
+ */
+ void CreateMultiPrecisionCopies() {
+for (auto const& stored_entry : store_) {
+ const int key = stored_entry.first;
+ const NDArray& stored = stored_entry.second;
+ if (stored.dtype() != mshadow::kFloat32) {
+auto& stored_realt = store_realt_[key];
+if (stored.storage_type() == kRowSparseStorage) {
+ stored_realt = NDArray(kRowSparseStorage, stored.shape(),
stored.ctx(),
+ true, mshadow::kFloat32);
+} else {
+ stored_realt = NDArray(stored.shape(), stored.ctx(), false,
mshadow::kFloat32);
+}
+
+auto& update = update_buf_[key];
+if (!update.merged.is_none()) {
+ update.merged = NDArray(update.merged.shape(), update.merged.ctx(),
false,
+ mshadow::kFloat32);
+}
+CHECK(update.request.size() == 0)
+ << ps::MyRank() << "Multiprecision mode can not be set while pushes
are underway."
+ << "Please set optimizer before pushing keys." << key << " " <<
update.request.size();
+
+CopyFromTo(stored, stored_realt);
+ }
+}
+for (auto const& stored_realt_entry : store_realt_) {
+ stored_realt_entry.second.WaitToRead();
+}
+ }
+
void DataHandleEx(const ps::KVMeta& req_meta,
-const ps::KVPairs& req_data,
-ps::KVServer* server) {
-DataHandleType recved_type = static_cast(req_meta.cmd);
-if (recved_type == DataHandleType::kRowSparsePushPull) {
- DataHandleRowSparse(req_meta, req_data, server);
-} else if (recved_type == DataHandleType::kCompressedPushPull) {
- DataHandleCompressed(req_meta, req_data, server);
-} else {
- DataHandleDefault(req_meta, req_data, server);
+const ps::KVPairs& req_data,
+ps::KVServer* server) {
+DataHandleType type = DepairDataHandleType(req_meta.cmd);
+switch (type.requestType) {
+ case RequestType::kRowSparsePushPull:
+DataHandleRowSparse(type, req_meta, req_data, server);
+break;
+ case RequestType::kCompressedPushPull:
+DataHandleCompressed(type, req_meta, req_data, server);
+break;
+ case RequestType::kDefaultPushPull:
+DataHandleDefault(type, req_meta, req_data, server);
+break;
}
-return;
}
- inline void ApplyUpdates(const int key, MergeBuf *merged, NDArray *stored,
- ps::KVServer* server) {
-if (merged->request.size() == (size_t) ps::NumWorkers()) {
+ inline bool has_multi_precision_copy(const DataHandleType type) {
+return multi_precision_ && type.dtype != mshadow::kFloat32;
+ }
+
+ inline void ApplyUpdates(const DataHandleType type, const int key,
+ UpdateBuf *updateBuf, ps::KVServer* server) {
+if (!sync_mode_ || updateBuf->request.size() == (size_t) ps::NumWorkers())
{
Review comment:
nit: `updateBuf ` -> `update_buf `?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10183: [MXNET-120] Float16 support for distributed training
eric-haibin-lin commented on a change in pull request #10183: [MXNET-120]
Float16 support for distributed training
URL: https://github.com/apache/incubator-mxnet/pull/10183#discussion_r178665130
##
File path: src/kvstore/kvstore_dist_server.h
##
@@ -170,43 +216,90 @@ class KVStoreDistServer {
app->Response(recved);
}
+ /*
+ * For keys already initialized, if necessary create stored_realt.
+ * This will only be used if by some wrong usage of kvstore,
+ * some keys are initialized before optimizer is set.
+ */
+ void CreateMultiPrecisionCopies() {
+for (auto const& stored_entry : store_) {
+ const int key = stored_entry.first;
+ const NDArray& stored = stored_entry.second;
+ if (stored.dtype() != mshadow::kFloat32) {
+auto& stored_realt = store_realt_[key];
+if (stored.storage_type() == kRowSparseStorage) {
+ stored_realt = NDArray(kRowSparseStorage, stored.shape(),
stored.ctx(),
+ true, mshadow::kFloat32);
Review comment:
nit: alignment
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10183: [MXNET-120] Float16 support for distributed training
eric-haibin-lin commented on a change in pull request #10183: [MXNET-120]
Float16 support for distributed training
URL: https://github.com/apache/incubator-mxnet/pull/10183#discussion_r178665347
##
File path: src/kvstore/kvstore_dist_server.h
##
@@ -170,43 +216,90 @@ class KVStoreDistServer {
app->Response(recved);
}
+ /*
+ * For keys already initialized, if necessary create stored_realt.
+ * This will only be used if by some wrong usage of kvstore,
+ * some keys are initialized before optimizer is set.
+ */
+ void CreateMultiPrecisionCopies() {
+for (auto const& stored_entry : store_) {
+ const int key = stored_entry.first;
+ const NDArray& stored = stored_entry.second;
+ if (stored.dtype() != mshadow::kFloat32) {
+auto& stored_realt = store_realt_[key];
+if (stored.storage_type() == kRowSparseStorage) {
+ stored_realt = NDArray(kRowSparseStorage, stored.shape(),
stored.ctx(),
+ true, mshadow::kFloat32);
+} else {
+ stored_realt = NDArray(stored.shape(), stored.ctx(), false,
mshadow::kFloat32);
+}
+
+auto& update = update_buf_[key];
+if (!update.merged.is_none()) {
+ update.merged = NDArray(update.merged.shape(), update.merged.ctx(),
false,
Review comment:
could `update.merged` also be sparse?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10183: [MXNET-120] Float16 support for distributed training
eric-haibin-lin commented on a change in pull request #10183: [MXNET-120] Float16 support for distributed training URL: https://github.com/apache/incubator-mxnet/pull/10183#discussion_r178646489 ## File path: python/mxnet/kvstore.py ## @@ -474,6 +474,8 @@ def set_optimizer(self, optimizer): except: raise self._send_command_to_servers(0, optim_str) +if optimizer.multi_precision: +self._send_command_to_servers(1, '') Review comment: Can we use meaningful CONST VAR instead of magic numbers? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10183: [MXNET-120] Float16 support for distributed training
eric-haibin-lin commented on a change in pull request #10183: [MXNET-120]
Float16 support for distributed training
URL: https://github.com/apache/incubator-mxnet/pull/10183#discussion_r178666240
##
File path: src/kvstore/kvstore_dist_server.h
##
@@ -220,175 +313,229 @@ class KVStoreDistServer {
}
}
- void DataHandleRowSparse(const ps::KVMeta& req_meta,
- const ps::KVPairs& req_data,
- ps::KVServer* server) {
+ void AccumulateRowSparseGrads(const DataHandleType type,
+const NDArray& recved,
+UpdateBuf* updateBuf) {
+NDArray out(kRowSparseStorage, updateBuf->merged.shape(), Context(), true,
+has_multi_precision_copy(type) ? mshadow::kFloat32 :
type.dtype);
+if (has_multi_precision_copy(type)) CopyFromTo(recved,
updateBuf->temp_array);
+const NDArray& to_merge = has_multi_precision_copy(type) ?
updateBuf->temp_array : recved;
+// accumulate row_sparse gradients
+// TODO(haibin) override + operator for row_sparse NDArray
+// instead of calling BinaryComputeRspRsp directly
+using namespace mshadow;
+Engine::Get()->PushAsync(
+[to_merge, updateBuf, out](RunContext ctx, Engine::CallbackOnComplete
on_complete) {
+ op::ElemwiseBinaryOp::ComputeEx(
+ {}, {}, {to_merge, updateBuf->merged}, {kWriteTo}, {out});
+ on_complete();
+}, to_merge.ctx(), {to_merge.var(), updateBuf->merged.var()}, {out.var()},
+FnProperty::kNormal, 0, PROFILER_MESSAGE_FUNCNAME);
+CopyFromTo(out, &(updateBuf->merged), 0);
+updateBuf->merged.WaitToRead();
+ }
+
+ void RowSparsePullResponse(const DataHandleType type,
+ const int master_key,
+ const size_t num_rows,
+ const ps::KVMeta& req_meta,
+ const ps::KVPairs& req_data,
+ ps::KVServer* server) {
+if (log_verbose_) LOG(INFO) << "pull: " << master_key;
+ps::KVPairs response;
+if (num_rows == 0) {
+ std::vector lens(req_data.keys.size(), 0);
+ response.keys = req_data.keys;
+ response.lens.CopyFrom(lens.begin(), lens.end());
+ server->Response(req_meta, response);
+ return;
+}
+const NDArray& stored = store_[master_key];
+if (has_multi_precision_copy(type)) stored.WaitToRead();
Review comment:
why need to wait on `stored`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction
cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction URL: https://github.com/apache/incubator-mxnet/issues/9632#issuecomment-378064596 https://github.com/apache/incubator-mxnet/pull/10375 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction
cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction URL: https://github.com/apache/incubator-mxnet/issues/9632#issuecomment-378064596 PR: https://github.com/apache/incubator-mxnet/pull/10375 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #10375: [MXNET-187] [WIP] fake shuffle functions
cjolivier01 commented on issue #10375: [MXNET-187] [WIP] fake shuffle functions URL: https://github.com/apache/incubator-mxnet/pull/10375#issuecomment-378064463 https://issues.apache.org/jira/browse/MXNET-187 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on issue #10366: fix bug in sgd
eric-haibin-lin commented on issue #10366: fix bug in sgd URL: https://github.com/apache/incubator-mxnet/pull/10366#issuecomment-378062143 Would setting MXNET_EXEC_NUM_TEMP help? @solin319 https://github.com/apache/incubator-mxnet/blob/master/docs/faq/env_var.md#memory-options This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu opened a new issue #10376: Flaky test_gluon.test_lambda
marcoabreu opened a new issue #10376: Flaky test_gluon.test_lambda
URL: https://github.com/apache/incubator-mxnet/issues/10376
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10373/1/pipeline
```
==
FAIL: test_gluon.test_lambda
--
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 197, in
runTest
self.test(*self.arg)
File "/work/mxnet/tests/python/unittest/common.py", line 157, in test_new
orig_test(*args, **kwargs)
File "/work/mxnet/tests/python/unittest/test_gluon.py", line 720, in
test_lambda
assert_almost_equal(out1.asnumpy(), out2.asnumpy(), rtol=1e-3)
File "/work/mxnet/python/mxnet/test_utils.py", line 493, in
assert_almost_equal
raise AssertionError(msg)
AssertionError:
Items are not equal:
Error 8.938683 exceeds tolerance rtol=0.001000, atol=0.00. Location of
maximum error:(0, 0, 1, 1), a=0.09, b=0.09
a: array(0.5227292 , 0.02360163, 0.42252338, ..., 0.26800904,
0.15513279, 0.59319496],
[0.59968483, 0.0948, 0.53139716, ..., 0.50179493,...
b: array(0.52272916, 0.02360159, 0.42252332, ..., 0.26800907,
0.15513274, 0.59319496],
[0.59968483, 0.0939, 0.5313972 , ..., 0.5017949 ,...
>> begin captured logging <<
common: INFO: Setting test np/mx/python random seeds, use
MXNET_TEST_SEED=703294434 to reproduce.
- >> end captured logging << -
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-mxnet] branch master updated: improve sparse.adagrad on GPU (#10312)
This is an automated email from the ASF dual-hosted git repository.
haibin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new eaa954c improve sparse.adagrad on GPU (#10312)
eaa954c is described below
commit eaa954c1a3ab92bfc3e783b79225025a77f93cca
Author: Haibin Lin
AuthorDate: Mon Apr 2 15:06:14 2018 -0700
improve sparse.adagrad on GPU (#10312)
---
src/operator/optimizer_op-inl.h | 35 +--
1 file changed, 33 insertions(+), 2 deletions(-)
diff --git a/src/operator/optimizer_op-inl.h b/src/operator/optimizer_op-inl.h
index 104f20a..c611a2b 100644
--- a/src/operator/optimizer_op-inl.h
+++ b/src/operator/optimizer_op-inl.h
@@ -1528,8 +1528,11 @@ inline bool AdagradStorageType(const nnvm::NodeAttrs&
attrs,
return dispatched;
}
+template
+struct AdagradDnsRspDnsKernel;
-struct AdagradDnsRspDnsKernel {
+template<>
+struct AdagradDnsRspDnsKernel {
template
MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
DType* state_data, const DType* weight_data, const IType* grad_idx,
@@ -1555,6 +1558,30 @@ struct AdagradDnsRspDnsKernel {
}
};
+template<>
+struct AdagradDnsRspDnsKernel {
+ template
+ MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
+DType* state_data, const DType* weight_data, const IType* grad_idx,
+const DType* grad_data, const DType clip_gradient, const DType epsilon,
+const DType lr, const DType rescale_grad) {
+using nnvm::dim_t;
+using namespace mshadow_op;
+const dim_t row_id = i / row_length;
+const dim_t col_id = i % row_length;
+const dim_t data_i = grad_idx[row_id] * row_length + col_id;
+DType grad_rescaled = grad_data[i] * rescale_grad;
+if (clip_gradient >= 0.0f) {
+ grad_rescaled = clip::Map(grad_rescaled, clip_gradient);
+}
+const DType grad_squared = grad_rescaled * grad_rescaled;
+state_data[data_i] += grad_squared;
+const DType div = grad_rescaled / square_root::Map(state_data[data_i] +
epsilon);
+// No need to use KERNEL_ASSIGN, as we already checked req is kWriteInplace
+out_data[data_i] = weight_data[data_i] - div * lr;
+ }
+};
+
template
void AdagradUpdateDnsRspDnsImpl(const AdagradParam& param,
const OpContext& ctx,
@@ -1582,7 +1609,11 @@ void AdagradUpdateDnsRspDnsImpl(const AdagradParam&
param,
DType* out_data = out->dptr();
const nnvm::dim_t nnr = grad.storage_shape()[0];
const auto row_length = weight.shape_.ProdShape(1, weight.ndim());
- Kernel::Launch(s, nnr, row_length,
+ size_t num_threads = nnr;
+ if (std::is_same::value) {
+num_threads = nnr * row_length;
+ }
+ Kernel, xpu>::Launch(s, num_threads,
row_length,
out_data, state_data, weight_data, grad_idx, grad_val,
static_cast(param.clip_gradient),
static_cast(param.epsilon),
static_cast(param.lr), static_cast(param.rescale_grad));
--
To stop receiving notification emails like this one, please contact
hai...@apache.org.
[GitHub] eric-haibin-lin closed pull request #10312: [MXNET-72] improve sparse adagrad on GPU
eric-haibin-lin closed pull request #10312: [MXNET-72] improve sparse adagrad
on GPU
URL: https://github.com/apache/incubator-mxnet/pull/10312
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/src/operator/optimizer_op-inl.h b/src/operator/optimizer_op-inl.h
index 104f20a61ee..c611a2b745d 100644
--- a/src/operator/optimizer_op-inl.h
+++ b/src/operator/optimizer_op-inl.h
@@ -1528,8 +1528,11 @@ inline bool AdagradStorageType(const nnvm::NodeAttrs&
attrs,
return dispatched;
}
+template
+struct AdagradDnsRspDnsKernel;
-struct AdagradDnsRspDnsKernel {
+template<>
+struct AdagradDnsRspDnsKernel {
template
MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
DType* state_data, const DType* weight_data, const IType* grad_idx,
@@ -1555,6 +1558,30 @@ struct AdagradDnsRspDnsKernel {
}
};
+template<>
+struct AdagradDnsRspDnsKernel {
+ template
+ MSHADOW_XINLINE static void Map(int i, index_t row_length, DType* out_data,
+DType* state_data, const DType* weight_data, const IType* grad_idx,
+const DType* grad_data, const DType clip_gradient, const DType epsilon,
+const DType lr, const DType rescale_grad) {
+using nnvm::dim_t;
+using namespace mshadow_op;
+const dim_t row_id = i / row_length;
+const dim_t col_id = i % row_length;
+const dim_t data_i = grad_idx[row_id] * row_length + col_id;
+DType grad_rescaled = grad_data[i] * rescale_grad;
+if (clip_gradient >= 0.0f) {
+ grad_rescaled = clip::Map(grad_rescaled, clip_gradient);
+}
+const DType grad_squared = grad_rescaled * grad_rescaled;
+state_data[data_i] += grad_squared;
+const DType div = grad_rescaled / square_root::Map(state_data[data_i] +
epsilon);
+// No need to use KERNEL_ASSIGN, as we already checked req is kWriteInplace
+out_data[data_i] = weight_data[data_i] - div * lr;
+ }
+};
+
template
void AdagradUpdateDnsRspDnsImpl(const AdagradParam& param,
const OpContext& ctx,
@@ -1582,7 +1609,11 @@ void AdagradUpdateDnsRspDnsImpl(const AdagradParam&
param,
DType* out_data = out->dptr();
const nnvm::dim_t nnr = grad.storage_shape()[0];
const auto row_length = weight.shape_.ProdShape(1, weight.ndim());
- Kernel::Launch(s, nnr, row_length,
+ size_t num_threads = nnr;
+ if (std::is_same::value) {
+num_threads = nnr * row_length;
+ }
+ Kernel, xpu>::Launch(s, num_threads,
row_length,
out_data, state_data, weight_data, grad_idx, grad_val,
static_cast(param.clip_gradient),
static_cast(param.epsilon),
static_cast(param.lr), static_cast(param.rescale_grad));
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rahul003 commented on issue #10283: [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon
rahul003 commented on issue #10283: [MXNET-242][Tutorial] Fine-tuning ONNX model in Gluon URL: https://github.com/apache/incubator-mxnet/pull/10283#issuecomment-378060200 The issue was likely with download, unable to reproduce the issue after fresh download. Sorry for the confusion. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] haojin2 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU
haojin2 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU URL: https://github.com/apache/incubator-mxnet/pull/10371#issuecomment-378059763 @anirudh2290 Corresponding documentations will be added once the implementations and tests are complete. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mbaijal commented on issue #10330: [Post 1.1][WIP] Couple of License Issues from 1.1 Release
mbaijal commented on issue #10330: [Post 1.1][WIP] Couple of License Issues from 1.1 Release URL: https://github.com/apache/incubator-mxnet/issues/10330#issuecomment-378059812 a. Fixed in PR #10373 b. Needs some investigation on best fix since we cant modify 3rdparty submodule This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nswamy commented on a change in pull request #10346: [MXNET-256] Add CI Test for GPU
nswamy commented on a change in pull request #10346: [MXNET-256] Add CI Test
for GPU
URL: https://github.com/apache/incubator-mxnet/pull/10346#discussion_r178661422
##
File path: ci/docker/runtime_functions.sh
##
@@ -427,6 +427,12 @@ unittest_ubuntu_cpu_scala() {
make scalatest USE_BLAS=openblas
}
+unittest_ubuntu_gpu_scala() {
+set -ex
+make scalapkg USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1
USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1
Review comment:
set a environment variable called SCALA_TEST_ON_GPU=1, in the test you can
check for this variable and change the context to gpu(0)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] anirudh2290 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU
anirudh2290 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU URL: https://github.com/apache/incubator-mxnet/pull/10371#issuecomment-378058175 Will dot(dns, csr) output a csr ndarray when cpu context is used, but will output dns ndarray is used ? This should be at least well documented somewhere. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] anirudh2290 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU
anirudh2290 commented on issue #10371: [MXNET-263] [WIP] Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on GPU URL: https://github.com/apache/incubator-mxnet/pull/10371#issuecomment-378058175 Will dot(dns, csr) output a csr ndarray when cpu context is used, but will output dns ndarray when gpu context is used ? This should be at least well documented somewhere. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 opened a new pull request #10375: [MXNET-187] [WIP] fake shuffle functions
cjolivier01 opened a new pull request #10375: [MXNET-187] [WIP] fake shuffle functions URL: https://github.com/apache/incubator-mxnet/pull/10375 ## Description ## (Brief description on what this PR is about) ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [ ] Feature1, tests, (and when applicable, API doc) - [ ] Feature2, tests, (and when applicable, API doc) ## Comments ## - If this change is a backward incompatible change, why must this change be made. - Interesting edge cases to note here This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] anirudh2290 commented on issue #10014: [MXNET-81] Fix crash with mx.nd.ones
anirudh2290 commented on issue #10014: [MXNET-81] Fix crash with mx.nd.ones URL: https://github.com/apache/incubator-mxnet/pull/10014#issuecomment-378055652 @piiswrong Thanks for your review! Is this good to merge ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] anirudh2290 opened a new pull request #10374: Sparse support for Custom Op
anirudh2290 opened a new pull request #10374: Sparse support for Custom Op URL: https://github.com/apache/incubator-mxnet/pull/10374 ## Description ## Adds sparse support for custom op. Registers InferStorageType and InferStorageTypeBackward interface for custom op. registers Forward and Backward with FStatefulComputeEx interface. Adds NDarray API to update chunk of a sparse ndarray from an existing ndarray. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [ ] Feature1, tests, (and when applicable, API doc) - [ ] Feature2, tests, (and when applicable, API doc) ## Comments ## - If this change is a backward incompatible change, why must this change be made. - Interesting edge cases to note here @piiswrong @eric-haibin-lin This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction
cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction URL: https://github.com/apache/incubator-mxnet/issues/9632#issuecomment-378048821 It appears that pytorch and TF don;t support Fermi GPUs. Do we wish to continue support? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction
cjolivier01 commented on issue #9632: Support pre-kepler GPUs without __shfl_down instruction URL: https://github.com/apache/incubator-mxnet/issues/9632#issuecomment-378048821 It appears that pytorch and TF don't support Fermi GPUs. Do we wish to continue support? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] zheng-da commented on issue #10317: [MXNET-264] Improve performance of MKLDNN in small batch sizes.
zheng-da commented on issue #10317: [MXNET-264] Improve performance of MKLDNN in small batch sizes. URL: https://github.com/apache/incubator-mxnet/pull/10317#issuecomment-378047340 Could you please review this PR? @piiswrong @pengzhao-intel @TaoLv This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 commented on issue #7848: __shfl_down is undefined. Is the end of CUDA 2.1 support?
cjolivier01 commented on issue #7848: __shfl_down is undefined. Is the end of CUDA 2.1 support? URL: https://github.com/apache/incubator-mxnet/issues/7848#issuecomment-378046738 Redirect to: https://github.com/apache/incubator-mxnet/issues/9632 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjolivier01 closed issue #7848: __shfl_down is undefined. Is the end of CUDA 2.1 support?
cjolivier01 closed issue #7848: __shfl_down is undefined. Is the end of CUDA 2.1 support? URL: https://github.com/apache/incubator-mxnet/issues/7848 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mbaijal opened a new pull request #10373: Adding a BSD file to LICENSE
mbaijal opened a new pull request #10373: Adding a BSD file to LICENSE URL: https://github.com/apache/incubator-mxnet/pull/10373 ## Description ## @eric-haibin-lin @marcoabreu Please review and merge Adding a file to top Level LICENSE file as per Issue #10330 part a and a couple of other fixes This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178645029 ## File path: python/mxnet/ndarray/sparse.py ## @@ -1159,6 +1185,293 @@ def _ndarray_cls(handle, writable=True, stype=_STORAGE_TYPE_UNDEFINED): _set_ndarray_class(_ndarray_cls) +def add(lhs, rhs): +"""Returns element-wise sum of the input arrays with broadcasting. + +Equivalent to ``lhs + rhs``, ``mx.nd.broadcast_add(lhs, rhs)`` and +``mx.nd.broadcast_plus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_add(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape.abs + +Parameters +-- +lhs : scalar or array +First array to be added. +rhs : scalar or array + Second array to be added. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape. + +Returns +--- +NDArray +The element-wise sum of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x+2).asnumpy() +array([[ 3., 3., 3.], + [ 3., 3., 3.]], dtype=float32) +>>> (x+y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> mx.nd.add(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> (z + y).asnumpy() +array([[ 0., 1.], + [ 1., 2.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_add, +operator.add, +_internal._plus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_add, +operator.add, +_internal._plus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def subtract(lhs, rhs): +"""Returns element-wise difference of the input arrays with broadcasting. + +Equivalent to ``lhs - rhs``, ``mx.nd.broadcast_sub(lhs, rhs)`` and +``mx.nd.broadcast_minus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_sub(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape. + +Parameters +-- +lhs : scalar or array +First array to be subtracted. +rhs : scalar or array + Second array to be subtracted. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape.__spec__ + +Returns +--- +NDArray +The element-wise difference of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x-2).asnumpy() +array([[-1., -1., -1.], + [-1., -1., -1.]], dtype=float32) +>>> (x-y).asnumpy() +array([[ 1., 1., 1.], + [ 0., 0., 0.]], dtype=float32) +>>> mx.nd.subtract(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 0., 0., 0.]], dtype=float32) +>>> (z-y).asnumpy() +array([[ 0., 1.], + [-1., 0.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_sub, +operator.sub, +_internal._minus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_sub, +operator.sub, +_internal._minus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def multiply(lhs, rhs): +"""Returns element-wise product of the input arrays with broadcasting. + +Equivalent to ``lhs * rhs`` and ``mx.nd.broadcast_mul(
[GitHub] eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178644856 ## File path: python/mxnet/ndarray/sparse.py ## @@ -1159,6 +1185,293 @@ def _ndarray_cls(handle, writable=True, stype=_STORAGE_TYPE_UNDEFINED): _set_ndarray_class(_ndarray_cls) +def add(lhs, rhs): +"""Returns element-wise sum of the input arrays with broadcasting. + +Equivalent to ``lhs + rhs``, ``mx.nd.broadcast_add(lhs, rhs)`` and +``mx.nd.broadcast_plus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_add(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape.abs + +Parameters +-- +lhs : scalar or array +First array to be added. +rhs : scalar or array + Second array to be added. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape. + +Returns +--- +NDArray +The element-wise sum of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x+2).asnumpy() +array([[ 3., 3., 3.], + [ 3., 3., 3.]], dtype=float32) +>>> (x+y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> mx.nd.add(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> (z + y).asnumpy() +array([[ 0., 1.], + [ 1., 2.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_add, +operator.add, +_internal._plus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_add, +operator.add, +_internal._plus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def subtract(lhs, rhs): +"""Returns element-wise difference of the input arrays with broadcasting. + +Equivalent to ``lhs - rhs``, ``mx.nd.broadcast_sub(lhs, rhs)`` and +``mx.nd.broadcast_minus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_sub(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape. + +Parameters +-- +lhs : scalar or array +First array to be subtracted. +rhs : scalar or array + Second array to be subtracted. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape.__spec__ + +Returns +--- +NDArray +The element-wise difference of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) Review comment: Same comment: use sparse ndarray in the example This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178644670 ## File path: python/mxnet/ndarray/sparse.py ## @@ -1159,6 +1185,293 @@ def _ndarray_cls(handle, writable=True, stype=_STORAGE_TYPE_UNDEFINED): _set_ndarray_class(_ndarray_cls) +def add(lhs, rhs): +"""Returns element-wise sum of the input arrays with broadcasting. + +Equivalent to ``lhs + rhs``, ``mx.nd.broadcast_add(lhs, rhs)`` and +``mx.nd.broadcast_plus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_add(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape.abs + +Parameters +-- +lhs : scalar or array +First array to be added. +rhs : scalar or array + Second array to be added. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape. + +Returns +--- +NDArray +The element-wise sum of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x+2).asnumpy() +array([[ 3., 3., 3.], + [ 3., 3., 3.]], dtype=float32) +>>> (x+y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> mx.nd.add(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> (z + y).asnumpy() +array([[ 0., 1.], + [ 1., 2.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_add, +operator.add, +_internal._plus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_add, +operator.add, +_internal._plus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def subtract(lhs, rhs): +"""Returns element-wise difference of the input arrays with broadcasting. + +Equivalent to ``lhs - rhs``, ``mx.nd.broadcast_sub(lhs, rhs)`` and +``mx.nd.broadcast_minus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_sub(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape. + +Parameters +-- +lhs : scalar or array +First array to be subtracted. +rhs : scalar or array + Second array to be subtracted. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape.__spec__ + +Returns +--- +NDArray +The element-wise difference of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x-2).asnumpy() +array([[-1., -1., -1.], + [-1., -1., -1.]], dtype=float32) +>>> (x-y).asnumpy() +array([[ 1., 1., 1.], + [ 0., 0., 0.]], dtype=float32) +>>> mx.nd.subtract(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 0., 0., 0.]], dtype=float32) +>>> (z-y).asnumpy() +array([[ 0., 1.], + [-1., 0.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_sub, +operator.sub, +_internal._minus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_sub, +operator.sub, +_internal._minus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def multiply(lhs, rhs): Review comment: Should these functions be added to `__all__` in line 36? -
[GitHub] eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
eric-haibin-lin commented on a change in pull request #10208: [MXNET-117]
Sparse operator broadcast_mul/div(csr, dense) = csr
URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178644551
##
File path: python/mxnet/ndarray/sparse.py
##
@@ -1159,6 +1185,293 @@ def _ndarray_cls(handle, writable=True,
stype=_STORAGE_TYPE_UNDEFINED):
_set_ndarray_class(_ndarray_cls)
+def add(lhs, rhs):
+"""Returns element-wise sum of the input arrays with broadcasting.
+
+Equivalent to ``lhs + rhs``, ``mx.nd.broadcast_add(lhs, rhs)`` and
+``mx.nd.broadcast_plus(lhs, rhs)`` when shapes of lhs and rhs do not
+match. If lhs.shape == rhs.shape, this is equivalent to
+``mx.nd.elemwise_add(lhs, rhs)``
+
+.. note::
+
+If the corresponding dimensions of two arrays have the same size or
one of them has size 1,
+then the arrays are broadcastable to a common shape.abs
+
+Parameters
+--
+lhs : scalar or array
+First array to be added.
+rhs : scalar or array
+ Second array to be added.
+If ``lhs.shape != rhs.shape``, they must be
+broadcastable to a common shape.
+
+Returns
+---
+NDArray
+The element-wise sum of the input arrays.
+
+Examples
+
+>>> x = mx.nd.ones((2,3))
Review comment:
I think we should replace the example with the supported case specifically
for sparse:
```
>>> x = mx.nd.ones((2,3)).tostype('csr')
>>> y = mx.nd.ones((2,3)).tostype('csr')
>>> (x + y).asnumpy()
array([[ 2., 2., 2.],
[ 2., 2., 2.]], dtype=float32)
>>> (x + 2).asnumpy()
array([[ 3., 3., 3.],
[ 3., 3., 3.]], dtype=float32)
>>> mx.nd.sparse.add(x,y).asnumpy()
array([[ 2., 2., 2.],
[ 2., 2., 2.]], dtype=float32)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178646005 ## File path: src/operator/tensor/elemwise_binary_broadcast_op_basic.cc ## @@ -120,8 +120,13 @@ Example:: broadcast_mul(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]] Review comment: Shall we also edit https://github.com/apache/incubator-mxnet/blob/master/docs/api/python/ndarray/sparse.md#arithmetic-operations and add broadcast_mul / broadcast_div? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lanking520 commented on issue #8132: How to disable MXNET_CUDNN_AUTOTUNE_DEFAULT and bucketing log message without turning off MXNET_CUDNN_AUTOTUNE_DEFAULT?
lanking520 commented on issue #8132: How to disable MXNET_CUDNN_AUTOTUNE_DEFAULT and bucketing log message without turning off MXNET_CUDNN_AUTOTUNE_DEFAULT? URL: https://github.com/apache/incubator-mxnet/issues/8132#issuecomment-378039215 @yxchng , is this still an issue for you? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr
eric-haibin-lin commented on a change in pull request #10208: [MXNET-117] Sparse operator broadcast_mul/div(csr, dense) = csr URL: https://github.com/apache/incubator-mxnet/pull/10208#discussion_r178645100 ## File path: python/mxnet/ndarray/sparse.py ## @@ -1159,6 +1185,293 @@ def _ndarray_cls(handle, writable=True, stype=_STORAGE_TYPE_UNDEFINED): _set_ndarray_class(_ndarray_cls) +def add(lhs, rhs): +"""Returns element-wise sum of the input arrays with broadcasting. + +Equivalent to ``lhs + rhs``, ``mx.nd.broadcast_add(lhs, rhs)`` and +``mx.nd.broadcast_plus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_add(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape.abs + +Parameters +-- +lhs : scalar or array +First array to be added. +rhs : scalar or array + Second array to be added. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape. + +Returns +--- +NDArray +The element-wise sum of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x+2).asnumpy() +array([[ 3., 3., 3.], + [ 3., 3., 3.]], dtype=float32) +>>> (x+y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> mx.nd.add(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 2., 2., 2.]], dtype=float32) +>>> (z + y).asnumpy() +array([[ 0., 1.], + [ 1., 2.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_add, +operator.add, +_internal._plus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_add, +operator.add, +_internal._plus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def subtract(lhs, rhs): +"""Returns element-wise difference of the input arrays with broadcasting. + +Equivalent to ``lhs - rhs``, ``mx.nd.broadcast_sub(lhs, rhs)`` and +``mx.nd.broadcast_minus(lhs, rhs)`` when shapes of lhs and rhs do not +match. If lhs.shape == rhs.shape, this is equivalent to +``mx.nd.elemwise_sub(lhs, rhs)`` + +.. note:: + +If the corresponding dimensions of two arrays have the same size or one of them has size 1, +then the arrays are broadcastable to a common shape. + +Parameters +-- +lhs : scalar or array +First array to be subtracted. +rhs : scalar or array + Second array to be subtracted. +If ``lhs.shape != rhs.shape``, they must be +broadcastable to a common shape.__spec__ + +Returns +--- +NDArray +The element-wise difference of the input arrays. + +Examples + +>>> x = mx.nd.ones((2,3)) +>>> y = mx.nd.arange(2).reshape((2,1)) +>>> z = mx.nd.arange(2).reshape((1,2)) +>>> x.asnumpy() +array([[ 1., 1., 1.], + [ 1., 1., 1.]], dtype=float32) +>>> y.asnumpy() +array([[ 0.], + [ 1.]], dtype=float32) +>>> z.asnumpy() +array([[ 0., 1.]], dtype=float32) +>>> (x-2).asnumpy() +array([[-1., -1., -1.], + [-1., -1., -1.]], dtype=float32) +>>> (x-y).asnumpy() +array([[ 1., 1., 1.], + [ 0., 0., 0.]], dtype=float32) +>>> mx.nd.subtract(x,y).asnumpy() +array([[ 1., 1., 1.], + [ 0., 0., 0.]], dtype=float32) +>>> (z-y).asnumpy() +array([[ 0., 1.], + [-1., 0.]], dtype=float32) +""" +# pylint: disable= no-member, protected-access +if isinstance(lhs, NDArray) and isinstance(rhs, NDArray) and lhs.shape == rhs.shape: +return _ufunc_helper( +lhs, +rhs, +op.elemwise_sub, +operator.sub, +_internal._minus_scalar, +None) + +return _ufunc_helper( +lhs, +rhs, +op.broadcast_sub, +operator.sub, +_internal._minus_scalar, +None) +# pylint: enable= no-member, protected-access + + +def multiply(lhs, rhs): Review comment: Let's show a specific example for csr multiplied by 1D vector
