[GitHub] pkuCactus edited a comment on issue #11868: nnpack_fully_connected-inl.h:45:55: error: expected template-name before ‘<’ token > class NNPACKFullyConnectedOp : public FullyConnectedOp
pkuCactus edited a comment on issue #11868: nnpack_fully_connected-inl.h:45:55:
error: expected template-name before ‘<’ token > class NNPACKFullyConnectedOp
: public FullyConnectedOp { >
^
URL:
https://github.com/apache/incubator-mxnet/issues/11868#issuecomment-457095164
@MartinRenaudin do you have resolved this, i've met the same error, could
you give some help? I found that in nnpack_fully_connected-inl.h the op inherit
from fullyconnectedop but it cannot be recognized since it cannot find in
fully_connected-inl.h. But i still have not method to resolve it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] pkuCactus commented on issue #11868: nnpack_fully_connected-inl.h:45:55: error: expected template-name before ‘<’ token > class NNPACKFullyConnectedOp : public FullyConnectedOp {
pkuCactus commented on issue #11868: nnpack_fully_connected-inl.h:45:55: error:
expected template-name before ‘<’ token > class NNPACKFullyConnectedOp :
public FullyConnectedOp { >
^
URL:
https://github.com/apache/incubator-mxnet/issues/11868#issuecomment-457095164
@MartinRenaudin do you have resolved this, i've met the same error, could
you give some help?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes
URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250486898
##
File path: example/gluon/wavenet/trainer.py
##
@@ -0,0 +1,130 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Module: WaveNet trainer modulep
+"""
+import sys
+import numpy as np
+import mxnet as mx
+from mxnet import gluon, autograd, nd
+from tqdm import trange
+
+from models import WaveNet
+from utils import decode_mu_law
+from data_loader import load_wav, data_generation, data_generation_sample
+# pylint: disable=invalid-name, too-many-arguments,
too-many-instance-attributes, no-member, no-self-use
+# set gpu count
+def setting_ctx(use_gpu):
+"""
+Description : setting cpu/gpu
+"""
+if eval(use_gpu):
+ctx = mx.gpu()
+else:
+ctx = mx.cpu()
+return ctx
+
+class Train():
+"""
+Description : Trainer for WaveNet
+"""
+def __init__(self, config):
+##setting hyper-parameters
+self.batch_size = config.batch_size
+self.epoches = config.epoches
+self.mu = config.mu
+self.n_residue = config.n_residue
+self.n_skip = config.n_skip
+self.dilation_depth = config.dilation_depth
+self.n_repeat = config.n_repeat
+self.seq_size = config.seq_size
+self.use_gpu = config.use_gpu
+self.ctx = setting_ctx(self.use_gpu)
+self.build_model()
+
+def build_model(self):
+"""
+Description : module for building network
+"""
+self.net = WaveNet(mu=self.mu, n_residue=self.n_residue,
n_skip=self.n_skip,\
+ dilation_depth=self.dilation_depth, n_repeat=self.n_repeat)
+#parameter initialization
+self.net.collect_params().initialize(ctx=self.ctx)
+#set optimizer
+self.trainer = gluon.Trainer(self.net.collect_params(),
optimizer='adam',\
+optimizer_params={'learning_rate':0.01})
+self.loss_fn = gluon.loss.SoftmaxCrossEntropyLoss()
+
+def save_model(self, epoch, current_loss):
+"""
+Description : module for saving network
+"""
+filename =
'models/best_perf_epoch_'+str(epoch)+"_loss_"+str(current_loss)
+self.net.save_params(filename)
+
+def train(self):
+"""
+Description : module for running train
+"""
+fs, data = load_wav('parametric-2.wav')
Review comment:
chage the wavefile name
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] lichun-wang commented on issue #12795: Deserialization problem with gluon `ValueError: There are multiple outputs with name ...`
lichun-wang commented on issue #12795: Deserialization problem with gluon `ValueError: There are multiple outputs with name ...` URL: https://github.com/apache/incubator-mxnet/issues/12795#issuecomment-457084807 i use mxnet 1.3.0 also meet this problem, In my code ,it was caused by code ' sym.get_internals()', after I deleted it , then it can run . This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin merged pull request #13346: Aggregate SGD
eric-haibin-lin merged pull request #13346: Aggregate SGD URL: https://github.com/apache/incubator-mxnet/pull/13346 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Aggregate SGD (#13346)
This is an automated email from the ASF dual-hosted git repository.
haibin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 0a45e1a Aggregate SGD (#13346)
0a45e1a is described below
commit 0a45e1a222637c7dee29511cbfc43e594571933b
Author: Przemyslaw Tredak
AuthorDate: Wed Jan 23 22:32:31 2019 -0800
Aggregate SGD (#13346)
* Aggregate SGD
* Make OpWrapperGenerator understand Tuple
* Trigger
* Add NNVM Tuple to cpp-package op.h
* Trigger
* Fix pylint aggregate SGD
* Update info about new ENV vars and modifying 2 tests that require
update_on_kvstore to be true
* Fix
* Aggregate SGD support for Gluon trainer
* Added text to doc about aggregate update in SGD optimizer
* Docs changes from review
---
cpp-package/scripts/OpWrapperGenerator.py | 4 +-
docs/faq/env_var.md | 9 +
python/mxnet/gluon/trainer.py | 15 +-
python/mxnet/model.py | 10 +-
python/mxnet/optimizer/optimizer.py | 231 --
src/operator/optimizer_op-inl.h | 295
src/operator/optimizer_op.cc| 193 +-
src/operator/optimizer_op.cu| 9 +
tests/python/unittest/test_gluon_trainer.py | 8 +-
tests/python/unittest/test_module.py| 3 +
10 files changed, 711 insertions(+), 66 deletions(-)
diff --git a/cpp-package/scripts/OpWrapperGenerator.py
b/cpp-package/scripts/OpWrapperGenerator.py
index ca430ec..65ba247 100644
--- a/cpp-package/scripts/OpWrapperGenerator.py
+++ b/cpp-package/scripts/OpWrapperGenerator.py
@@ -97,7 +97,8 @@ class Arg:
'double':'double',\
'double or None':'dmlc::optional',\
'Shape or None':'dmlc::optional',\
-'string':'const std::string&'}
+'string':'const std::string&',\
+'tuple of ':'nnvm::Tuple'}
name = ''
type = ''
description = ''
@@ -407,6 +408,7 @@ if __name__ == "__main__":
"#include \"mxnet-cpp/op_util.h\"\n"
"#include \"mxnet-cpp/operator.h\"\n"
"#include \"dmlc/optional.h\"\n"
+ "#include \"nnvm/tuple.h\"\n"
"\n"
"namespace mxnet {\n"
"namespace cpp {\n"
diff --git a/docs/faq/env_var.md b/docs/faq/env_var.md
index 98057d0..99ebae2 100644
--- a/docs/faq/env_var.md
+++ b/docs/faq/env_var.md
@@ -145,6 +145,10 @@ $env:MXNET_STORAGE_FALLBACK_LOG_VERBOSE=0
- If true, MXNet tries to use GPU peer-to-peer communication, if available
on your device,
when kvstore's type is `device`.
+* MXNET_UPDATE_ON_KVSTORE
+ - Values: 0(false) or 1(true) ```(default=1)```
+ - If true, weight updates are performed during the communication step, if
possible.
+
## Memonger
* MXNET_BACKWARD_DO_MIRROR
@@ -218,6 +222,11 @@ When USE_PROFILER is enabled in Makefile or CMake, the
following environments ca
- When the array size is bigger than or equal to this threshold,
NDArray::Copy(from, to) is implemented by OpenMP with the Recommended OMP
Thread Count.
- When the array size is less than this threshold, NDArray::Copy(from , to))
is implemented by memcpy in single thread.
+* MXNET_OPTIMIZER_AGGREGATION_SIZE
+ - Values: Int ```(default=4)```
+ - Maximum value is 60.
+ - This variable controls how many weights will be updated in a single call
to optimizer (for optimizers that support aggregation, currently limited to
SGD).
+
Settings for Minimum Memory Usage
-
- Make sure ```min(MXNET_EXEC_NUM_TEMP, MXNET_GPU_WORKER_NTHREADS) = 1```
diff --git a/python/mxnet/gluon/trainer.py b/python/mxnet/gluon/trainer.py
index f6c0a31..8060f38 100644
--- a/python/mxnet/gluon/trainer.py
+++ b/python/mxnet/gluon/trainer.py
@@ -60,7 +60,8 @@ class Trainer(object):
See mxnet.KVStore.set_gradient_compression method for more details on
gradient compression.
update_on_kvstore : bool, default None
Whether to perform parameter updates on kvstore. If None, then trainer
will choose the more
-suitable option depending on the type of kvstore.
+suitable option depending on the type of kvstore. If the
`update_on_kvstore` argument is
+provided, environment variable `MXNET_UPDATE_ON_KVSTORE` will be
ignored.
Properties
--
@@ -393,6 +394,8 @@ class Trainer(object):
self._update(ignore_stale_grad)
def _update(self, ignore_stale_grad=False):
+updates = [[] for _ in self._updaters]
+
for i, param in enumerate(self._params):
if param.grad_req == 'null':
continue
@@ -416,11 +419,17 @@ class Trainer(object):
[GitHub] marcoabreu commented on a change in pull request #13964: Addresses comments in runtime feature discovery API
marcoabreu commented on a change in pull request #13964: Addresses comments in runtime feature discovery API URL: https://github.com/apache/incubator-mxnet/pull/13964#discussion_r250477268 ## File path: include/mxnet/c_api.h ## @@ -208,6 +208,24 @@ MXNET_DLL const char *MXGetLastError(); //- // Part 0: Global State setups //- + +/*! + * \brief Check if a feature is enabled in the runtime + * \param feature to check mxruntime.h + * \param out set to true if the feature is enabled, false otherwise + * \return 0 when success, -1 when failure happens. + */ +MXNET_DLL int MXRuntimeHasFeature(const mx_uint feature, bool *out); Review comment: As far as I understand, this function is not intended for usage within high performance loops, so I would rather prefer usability instead of optimizing for a few bytes of allocation. I'm also in favour of a dictionary with the function name as key and the status as boolean value. For HPC cases, you could still cache the results if really necessary This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Flaky maven binary download (#13974)
This is an automated email from the ASF dual-hosted git repository. marcoabreu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new a19b0b9 Flaky maven binary download (#13974) a19b0b9 is described below commit a19b0b9af498974931614c38cb9d807fc89a8f03 Author: Zach Kimberg AuthorDate: Wed Jan 23 22:09:44 2019 -0800 Flaky maven binary download (#13974) --- ci/docker/install/centos7_scala.sh | 5 - ci/docker/install/ubuntu_publish.sh | 7 ++- ci/docker/install/ubuntu_scala.sh | 4 +++- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/ci/docker/install/centos7_scala.sh b/ci/docker/install/centos7_scala.sh index 5c43f01..5a1c416 100755 --- a/ci/docker/install/centos7_scala.sh +++ b/ci/docker/install/centos7_scala.sh @@ -25,8 +25,11 @@ set -ex yum install -y java-1.8.0-openjdk-devel export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk export PATH=$JAVA_HOME/bin:$PATH + # Build from source with Maven -wget -q http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz +curl -o apache-maven-3.3.9-bin.tar.gz http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz \ +|| curl -o apache-maven-3.3.9-bin.tar.gz https://search.maven.org/remotecontent?filepath=org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.tar.gz + tar xzf apache-maven-3.3.9-bin.tar.gz mkdir /usr/local/maven mv apache-maven-3.3.9/ /usr/local/maven/ diff --git a/ci/docker/install/ubuntu_publish.sh b/ci/docker/install/ubuntu_publish.sh index 1ad6ab9..1fb7bf1 100755 --- a/ci/docker/install/ubuntu_publish.sh +++ b/ci/docker/install/ubuntu_publish.sh @@ -18,6 +18,8 @@ # under the License. # Build on Ubuntu 14.04 LTS for LINUX CPU/GPU +set -ex + apt-get update apt-get install -y software-properties-common add-apt-repository ppa:ubuntu-toolchain-r/test -y @@ -45,7 +47,10 @@ apt-get install -y git \ automake \ pkg-config \ openjdk-8-jdk -curl -o apache-maven-3.3.9-bin.tar.gz http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz + +curl -o apache-maven-3.3.9-bin.tar.gz http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz \ +|| curl -o apache-maven-3.3.9-bin.tar.gz https://search.maven.org/remotecontent?filepath=org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.tar.gz + tar xzf apache-maven-3.3.9-bin.tar.gz mkdir /usr/local/maven mv apache-maven-3.3.9/ /usr/local/maven/ diff --git a/ci/docker/install/ubuntu_scala.sh b/ci/docker/install/ubuntu_scala.sh index 8b23d61..9115bbc 100755 --- a/ci/docker/install/ubuntu_scala.sh +++ b/ci/docker/install/ubuntu_scala.sh @@ -40,7 +40,9 @@ apt-get install -y \ # Ubuntu 14.04 if [[ $(lsb_release -r | grep 14.04) ]]; then -curl -o apache-maven-3.3.9-bin.tar.gz http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz +curl -o apache-maven-3.3.9-bin.tar.gz http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz \ +|| curl -o apache-maven-3.3.9-bin.tar.gz https://search.maven.org/remotecontent?filepath=org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.tar.gz + tar xzf apache-maven-3.3.9-bin.tar.gz mkdir /usr/local/maven mv apache-maven-3.3.9/ /usr/local/maven/
[GitHub] marcoabreu merged pull request #13974: Fix flaky maven binary download
marcoabreu merged pull request #13974: Fix flaky maven binary download URL: https://github.com/apache/incubator-mxnet/pull/13974 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] apeforest commented on issue #13980: [WIP] Use CPUPinned context in ImageRecordIOParser2
apeforest commented on issue #13980: [WIP] Use CPUPinned context in ImageRecordIOParser2 URL: https://github.com/apache/incubator-mxnet/pull/13980#issuecomment-457077234 @eric-haibin-lin Could you please help to review this PR. You have raised some concerns on this change in #12666 earlier. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes
URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250475542
##
File path: example/gluon/wavenet/trainer.py
##
@@ -0,0 +1,130 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Module: WaveNet trainer modulep
+"""
+import sys
+import numpy as np
+import mxnet as mx
+from mxnet import gluon, autograd, nd
+from tqdm import trange
+
+from models import WaveNet
+from utils import decode_mu_law
+from data_loader import load_wav, data_generation, data_generation_sample
+# pylint: disable=invalid-name, too-many-arguments,
too-many-instance-attributes, no-member, no-self-use
+# set gpu count
+def setting_ctx(use_gpu):
+"""
+Description : setting cpu/gpu
+"""
+if eval(use_gpu):
+ctx = mx.gpu()
+else:
+ctx = mx.cpu()
+return ctx
+
+class Train():
+"""
+Description : Trainer for WaveNet
+"""
+def __init__(self, config):
+##setting hyper-parameters
+self.batch_size = config.batch_size
+self.epoches = config.epoches
+self.mu = config.mu
+self.n_residue = config.n_residue
+self.n_skip = config.n_skip
+self.dilation_depth = config.dilation_depth
+self.n_repeat = config.n_repeat
+self.seq_size = config.seq_size
+self.use_gpu = config.use_gpu
+self.ctx = setting_ctx(self.use_gpu)
+self.build_model()
+
+def build_model(self):
+"""
+Description : module for building network
+"""
+self.net = WaveNet(mu=self.mu, n_residue=self.n_residue,
n_skip=self.n_skip,\
+ dilation_depth=self.dilation_depth, n_repeat=self.n_repeat)
+#parameter initialization
+self.net.collect_params().initialize(ctx=self.ctx)
+#set optimizer
+self.trainer = gluon.Trainer(self.net.collect_params(),
optimizer='adam',\
+optimizer_params={'learning_rate':0.01})
+self.loss_fn = gluon.loss.SoftmaxCrossEntropyLoss()
+
+def save_model(self, epoch, current_loss):
+"""
+Description : module for saving network
+"""
+filename =
'models/best_perf_epoch_'+str(epoch)+"_loss_"+str(current_loss)
+self.net.save_params(filename)
Review comment:
Change completed
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes
URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250474419
##
File path: example/gluon/wavenet/models.py
##
@@ -0,0 +1,118 @@
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Module: WaveNet network modulep
+"""
+from mxnet import nd
+from mxnet.gluon import nn
+import mxnet.ndarray as F
+# pylint: disable=invalid-name, too-many-arguments, arguments-differ,
attribute-defined-outside-init, too-many-instance-attributes,
invalid-sequence-index, no-self-use
+class One_Hot(nn.Block):
+"""
+Description : generate one hot result
+"""
+def __init__(self, depth):
+super(One_Hot, self).__init__()
+self.depth = depth
+
+def forward(self, X_in):
+with X_in.context:
+X_in = X_in
+self.ones = nd.one_hot(nd.arange(self.depth), self.depth)
+return self.ones[X_in, :]
+
+def __repr__(self):
+return self.__class__.__name__ + "({})".format(self.depth)
+
+class WaveNet(nn.Block):
+"""
+mu: audio quantization size
+n_residue: residue channels
+n_skip: skip channels
+dilation_depth & n_repeat: dilation layer setup
Review comment:
Delete & and add new line
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] yuxihu commented on issue #13980: [WIP] Use CPUPinned context in ImageRecordIOParser2
yuxihu commented on issue #13980: [WIP] Use CPUPinned context in ImageRecordIOParser2 URL: https://github.com/apache/incubator-mxnet/pull/13980#issuecomment-457076602 @mxnet-label-bot update [pr-work-in-progress] This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] yuxihu opened a new pull request #13980: [WIP] Use CPUPinned context in ImageRecordIOParser2
yuxihu opened a new pull request #13980: [WIP] Use CPUPinned context in ImageRecordIOParser2 URL: https://github.com/apache/incubator-mxnet/pull/13980 This PR is to fix performance regression introduced by [#12666](https://github.com/apache/incubator-mxnet/pull/12666). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250474071 ## File path: example/gluon/wavenet/models.py ## @@ -0,0 +1,118 @@ + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Module: WaveNet network modulep +""" +from mxnet import nd +from mxnet.gluon import nn +import mxnet.ndarray as F +# pylint: disable=invalid-name, too-many-arguments, arguments-differ, attribute-defined-outside-init, too-many-instance-attributes, invalid-sequence-index, no-self-use +class One_Hot(nn.Block): Review comment: delete this line This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes
URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250473457
##
File path: example/gluon/wavenet/main.py
##
@@ -0,0 +1,48 @@
+"""
+Descrition : main module to run code
+"""
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import argparse
+from trainer import Train
+
+def main():
+"""
+Description : run code using argument info
+"""
+parser = argparse.ArgumentParser()
+parser = argparse.ArgumentParser()
+parser.add_argument('--batch_size', type=int, default=64)
Review comment:
Add description to each argument
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250469965 ## File path: example/gluon/wavenet/data_loader.py ## @@ -0,0 +1,60 @@ +""" +Description : Set DataSet module for Wavenet +""" +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License +import os Review comment: complete changing order This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250469965 ## File path: example/gluon/wavenet/data_loader.py ## @@ -0,0 +1,60 @@ +""" +Description : Set DataSet module for Wavenet +""" +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License +import os Review comment: change order This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes
URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250469052
##
File path: example/gluon/wavenet/main.py
##
@@ -0,0 +1,48 @@
+"""
+Descrition : main module to run code
+"""
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import argparse
+from trainer import Train
+
+def main():
+"""
+Description : run code using argument info
+"""
+parser = argparse.ArgumentParser()
+parser = argparse.ArgumentParser()
+parser.add_argument('--batch_size', type=int, default=64)
+parser.add_argument('--epoches', type=int, default=10)
+parser.add_argument('--mu', type=int, default=128)
+parser.add_argument('--n_residue', type=int, default=24)
+parser.add_argument('--n_skip', type=int, default=128)
+parser.add_argument('--dilation_depth', type=int, default=10)
+parser.add_argument('--n_repeat', type=int, default=2)
+parser.add_argument('--seq_size', type=int, default=2)
+parser.add_argument('--use_gpu', type=str, default="True")
+parser.add_argument('--generation', type=bool, default=True)
+config = parser.parse_args()
+
Review comment:
add this code
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes
URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250468853
##
File path: example/gluon/wavenet/main.py
##
@@ -0,0 +1,48 @@
+"""
+Descrition : main module to run code
+"""
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import argparse
+from trainer import Train
+
+def main():
+"""
+Description : run code using argument info
+"""
+parser = argparse.ArgumentParser()
+parser = argparse.ArgumentParser()
+parser.add_argument('--batch_size', type=int, default=64)
+parser.add_argument('--epoches', type=int, default=10)
Review comment:
fix typos
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250468519 ## File path: example/gluon/wavenet/main.py ## @@ -0,0 +1,48 @@ +""" +Descrition : main module to run code +""" +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +import argparse +from trainer import Train + +def main(): +""" +Description : run code using argument info +""" +parser = argparse.ArgumentParser() +parser = argparse.ArgumentParser() Review comment: delete duplicated line This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250468397 ## File path: example/gluon/wavenet/data_loader.py ## @@ -0,0 +1,60 @@ +""" +Description : Set DataSet module for Wavenet Review comment: Fixed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13735: update wavenet codes
seujung commented on a change in pull request #13735: update wavenet codes URL: https://github.com/apache/incubator-mxnet/pull/13735#discussion_r250467423 ## File path: example/gluon/wavenet/README.md ## @@ -0,0 +1,53 @@ +# WaveNet with Gluon + +Gluon implementation of [WaveNet: A Generative Model for Raw Audio](https://arxiv.org/abs/1609.03499) + + + + +## Requirements +- Python 3.6.1 +- Mxnet 1.2 +- tqdm +- scipy.io + + +## Usage + +- arguments + - batch_size : Define batch size (defualt=64) + - epochs : Define the total epoches (default=1000) + - mu : Define mu value for [mu-law algorithm](https://en.wikipedia.org/wiki/%CE%9C-law_algorithm) (default=128) + - n_residue : Define number of residue (default=24) + - dilation_depth : Define dilation depth (default=10) + - use_gpu : whether or not to use the GPU (default=True) + - generation : whether or not to generate a wave file for model (default=True) + +## default setting +``` +python main.py +``` +or + +## manual setting +``` +python main.py --batch_size=32 --epoches=100 ... Review comment: fix typos This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seujung commented on a change in pull request #13647: Update lip reading example
seujung commented on a change in pull request #13647: Update lip reading example
URL: https://github.com/apache/incubator-mxnet/pull/13647#discussion_r250466069
##
File path: example/gluon/lipnet/lipnet_model.ipynb
##
@@ -0,0 +1,249 @@
+{
Review comment:
I think running main.py is a good way to run tis example. So is it possible
to delete the notebook file?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] soeque1 commented on a change in pull request #13647: Update lip reading example
soeque1 commented on a change in pull request #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#discussion_r250465119 ## File path: example/gluon/lipnet/models/network.py ## @@ -0,0 +1,78 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Description : LipNet module using gluon +""" + + +from mxnet import nd +from mxnet.gluon import nn, rnn +import mxnet.ndarray as F +# pylint: disable=too-many-instance-attributes +class LipNet(nn.Block): +""" +Description : LipNet network using gluon +""" +def __init__(self, dr_rate, **kwargs): +super(LipNet, self).__init__(**kwargs) +with self.name_scope(): +self.conv1 = nn.Conv3D(32, kernel_size=(3, 5, 5), strides=(1, 2, 2), padding=(1, 2, 2)) +#self.bn1 = nn.BatchNorm() +self.bn1 = nn.InstanceNorm(in_channels=32) +self.dr1 = nn.Dropout(dr_rate) +self.pool1 = nn.MaxPool3D((1, 2, 2), (1, 2, 2)) +self.conv2 = nn.Conv3D(64, kernel_size=(3, 5, 5), strides=(1, 1, 1), padding=(1, 2, 2)) +#self.bn2 = nn.BatchNorm() +self.bn2 = nn.InstanceNorm(in_channels=64) +self.dr2 = nn.Dropout(dr_rate) +self.pool2 = nn.MaxPool3D((1, 2, 2), (1, 2, 2)) +self.conv3 = nn.Conv3D(96, kernel_size=(3, 3, 3), strides=(1, 1, 1), padding=(1, 2, 2)) +#self.bn3 = nn.BatchNorm() +self.bn3 = nn.InstanceNorm(in_channels=96) +self.dr3 = nn.Dropout(dr_rate) +self.pool3 = nn.MaxPool3D((1, 2, 2), (1, 2, 2)) +self.gru1 = rnn.GRU(256, bidirectional=True) +self.gru2 = rnn.GRU(256, bidirectional=True) +self.dense = nn.Dense(27+1, flatten=False) +# pylint: disable=arguments-differ +def forward(self, x): +out = self.conv1(x) +out = self.bn1(out) +out = F.relu(out) +out = self.dr1(out) +out = self.pool1(out) +out = self.conv2(out) +out = self.bn2(out) +out = F.relu(out) +out = self.dr2(out) +out = self.pool2(out) +out = self.conv3(out) +out = self.bn3(out) +out = F.relu(out) +out = self.dr3(out) +out = self.pool3(out) +out = nd.transpose(out, (2, 0, 1, 3, 4)) +# pylint: disable=no-member +out = out.reshape((out.shape[0], out.shape[1], -1)) Review comment: I didn't konw what 0 means. So I changed the code. ``` out = out.reshape((self.seq_len, self.batch_size, -1)) ``` It also works when the code is changed like this. ``` out = out.reshape((0, 0, -1)) ``` If you want to change ```0```, I will do. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] soeque1 commented on a change in pull request #13647: Update lip reading example
soeque1 commented on a change in pull request #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#discussion_r250465435 ## File path: example/gluon/lipnet/models/network.py ## @@ -0,0 +1,78 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Description : LipNet module using gluon +""" + + +from mxnet import nd +from mxnet.gluon import nn, rnn +import mxnet.ndarray as F Review comment: Updated This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ChaiBapchya commented on issue #13855: Fix symbolic example for rand_zipfian function
ChaiBapchya commented on issue #13855: Fix symbolic example for rand_zipfian function URL: https://github.com/apache/incubator-mxnet/pull/13855#issuecomment-457063320 @safrooze @ThomasDelteil Do you mind considering my PR since I had discovered it before and made PR as well? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] soeque1 commented on a change in pull request #13647: Update lip reading example
soeque1 commented on a change in pull request #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#discussion_r250463199 ## File path: example/gluon/lipnet/models/network.py ## @@ -0,0 +1,78 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Description : LipNet module using gluon +""" + + +from mxnet import nd +from mxnet.gluon import nn, rnn +import mxnet.ndarray as F +# pylint: disable=too-many-instance-attributes +class LipNet(nn.Block): Review comment: Hybridized This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] srochel commented on issue #13713: MXNet usage on non-AWS cloud providers
srochel commented on issue #13713: MXNet usage on non-AWS cloud providers URL: https://github.com/apache/incubator-mxnet/issues/13713#issuecomment-457062336 Hi Aaron - I provide information about the NV VM's in the links. What additional content are you looking for? We can ping our friends from NV for more details. Let me know. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eureka7mt commented on issue #13612: add pos_weight for SigmoidBinaryCrossEntropyLoss
eureka7mt commented on issue #13612: add pos_weight for SigmoidBinaryCrossEntropyLoss URL: https://github.com/apache/incubator-mxnet/pull/13612#issuecomment-457052992 Don't know why it failed in test_multinomial_generator() which is in the file '/work/mxnet/tests/python/gpu/../unittest/test_random.py' with unix-gpu. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] yifeim commented on issue #13244: hybridize rnn and add model graph
yifeim commented on issue #13244: hybridize rnn and add model graph URL: https://github.com/apache/incubator-mxnet/pull/13244#issuecomment-457043397 @sandeep-krishnamurthy Thanks for the reviews so far. I agree with you but I am very confused what is exactly needed here. Please let me know how I can help. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] soeque1 commented on a change in pull request #13647: Update lip reading example
soeque1 commented on a change in pull request #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#discussion_r250447022 ## File path: example/gluon/lipnet/trainer.py ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Description : Training module for LipNet +""" + + +import sys +import mxnet as mx +from mxnet import gluon, autograd, nd +from mxnet.gluon.data.vision import transforms +from tqdm import tqdm, trange +from data_loader import LipsDataset +from models.network import LipNet +from BeamSearch import ctcBeamSearch +from utils.common import char_conv, int2char +# set gpu count + + +def setting_ctx(num_gpus): +""" +Description : set gpu module +""" +if num_gpus > 0: +ctx = [mx.gpu(i) for i in range(num_gpus)] +else: +ctx = [mx.cpu()] +return ctx + + +ALPHABET = '' +for i in range(27): +ALPHABET += int2char(i) + +def char_beam_search(out): +""" +Description : apply beam search for prediction result +""" +out_conv = list() +for idx in range(out.shape[0]): +probs = out[idx] +prob = probs.softmax().asnumpy() +line_string_proposals = ctcBeamSearch(prob, ALPHABET, None, k=4, beamWidth=25) +out_conv.append(line_string_proposals[0]) +return out_conv + +# pylint: disable=too-many-instance-attributes, too-many-locals +class Train: +""" +Description : Train class for training network +""" +def __init__(self, config): +##setting hyper-parameters +self.batch_size = config.batch_size +self.epochs = config.epochs +self.image_path = config.image_path +self.align_path = config.align_path +self.dr_rate = config.dr_rate +self.num_gpus = config.num_gpus +self.ctx = setting_ctx(self.num_gpus) +self.num_workers = config.num_workers +self.build_model() + +def build_model(self): +""" +Description : build network +""" +#set network +self.net = LipNet(self.dr_rate) +self.net.initialize(ctx=self.ctx) +#set optimizer +self.loss_fn = gluon.loss.CTCLoss() +self.trainer = gluon.Trainer(self.net.collect_params(), \ + optimizer='SGD') +def save_model(self, epoch, iter_no, current_loss): +""" +Description : save parameter of network weight +""" +file_name = "checkpoint/best_model_epoches_"+str(epoch)+"iter_"+str(iter_no)+ \ +"loss_"+str(round(current_loss, 2)) +self.net.save_parameters(file_name) + +def train(self): +""" +Description : training for LipNet +""" +input_transform = transforms.Compose([transforms.ToTensor(), \ + transforms.Normalize((0.7136, 0.4906, 0.3283), \ + (0.1138, 0.1078, 0.0917))]) +training_dataset = LipsDataset(self.image_path, self.align_path, transform=input_transform) +train_dataloader = mx.gluon.data.DataLoader(training_dataset, +batch_size=self.batch_size, +shuffle=True, + num_workers=self.num_workers) +best_loss = sys.maxsize +for epoch in trange(self.epochs): +iter_no = 0 +for input_data, input_label in tqdm(train_dataloader): +input_data = nd.transpose(input_data, (0, 2, 1, 3, 4)) Review comment: Moved into data_loader This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] soeque1 commented on a change in pull request #13647: Update lip reading example
soeque1 commented on a change in pull request #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#discussion_r250446948 ## File path: example/gluon/lipnet/data_loader.py ## @@ -0,0 +1,73 @@ +""" +Description : Set DataSet module for lip images +""" +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import os +import glob +from mxnet import nd +import mxnet.gluon.data.dataset as dataset +from mxnet.gluon.data.vision.datasets import image +from utils.align import Align + +class LipsDataset(dataset.Dataset): +""" +Description : DataSet class for lip images +""" +def __init__(self, root, align_root, flag=1, transform=None): +self._root = os.path.expanduser(root) +self._align_root = align_root +self._flag = flag +self._transform = transform +self._exts = ['.jpg', '.jpeg', '.png'] +self._list_images(self._root) + +def _list_images(self, root): +""" +Description : generate list for lip images +""" +self.labels = [] +self.items = [] +folder_path = glob.glob(os.path.join(root, "*", "*")) +for folder in folder_path: +filename = glob.glob(os.path.join(folder, "*")) +if len(filename) != 75: +continue +filename.sort() +label = os.path.split(folder)[-1] +self.items.append((filename, label)) +def align_generation(self, file_nm, padding=75): +""" +Description : Align to lip position +""" +align = Align(self._align_root + '/' + file_nm + '.align') +return nd.array(align.sentence(padding)) +def __getitem__(self, idx): +img = list() +for image_name in self.items[idx][0]: +tmp_img = image.imread(image_name, self._flag) +if self._transform is not None: +tmp_img = self._transform(tmp_img) +img.append(tmp_img) +img = nd.stack(*img) +#print(self.items[idx][0][0]) Review comment: Removed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] samskalicky commented on issue #13976: Problem of exporting FP16 SyncBN model.
samskalicky commented on issue #13976: Problem of exporting FP16 SyncBN model.
URL:
https://github.com/apache/incubator-mxnet/issues/13976#issuecomment-457033687
Thanks @Fiend1213, i tried commenting out this line:
```
net.cast('float16')
```
and then change this line:
```
outputs = net(X.astype('float16'))
outputs = net(X)
```
and the script completed. Obviously this means its not doing it in float16,
but at least its succeeding. Im building from source using your build flags
now, will try rerunning and debugging in to try see what the issue is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] lanking520 commented on issue #13632: Gradient multiplier (contrib) operator
lanking520 commented on issue #13632: Gradient multiplier (contrib) operator URL: https://github.com/apache/incubator-mxnet/pull/13632#issuecomment-457032564 @szha @ThomasDelteil merge? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #7359: Compile MXNet error: no matching function for call to ‘std::vector::push_back
ThomasDelteil closed issue #7359: Compile MXNet error: no matching function for call to ‘std::vector::push_back URL: https://github.com/apache/incubator-mxnet/issues/7359 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #13410: ARM QEMU test in CI failed unrelated PR
ThomasDelteil closed issue #13410: ARM QEMU test in CI failed unrelated PR URL: https://github.com/apache/incubator-mxnet/issues/13410 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #9742: Feature request: please translate zh.gluon.ai in English
ThomasDelteil closed issue #9742: Feature request: please translate zh.gluon.ai in English URL: https://github.com/apache/incubator-mxnet/issues/9742 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #9742: Feature request: please translate zh.gluon.ai in English
ThomasDelteil commented on issue #9742: Feature request: please translate zh.gluon.ai in English URL: https://github.com/apache/incubator-mxnet/issues/9742#issuecomment-457028041 available here: http://d2l.ai This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.
This is an automated email from the ASF dual-hosted git repository. zhasheng pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 8e167a4 Bump the publish timestamp. 8e167a4 is described below commit 8e167a4c747b3060aa1b0468e9fbb207b1c1f857 Author: mxnet-ci AuthorDate: Thu Jan 24 00:59:28 2019 + Bump the publish timestamp. --- date.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/date.txt b/date.txt new file mode 100644 index 000..653ffec --- /dev/null +++ b/date.txt @@ -0,0 +1 @@ +Thu Jan 24 00:59:28 UTC 2019
[GitHub] lanking520 opened a new pull request #13979: [MXNET-1232] fix demo and add Eclipse support
lanking520 opened a new pull request #13979: [MXNET-1232] fix demo and add Eclipse support URL: https://github.com/apache/incubator-mxnet/pull/13979 ## Description ## Get rid of makefile and add Eclipse support for Java @andrewfayres @piyushghai @zachgk @aaronmarkham ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Fiend1213 commented on issue #13976: Problem of exporting FP16 SyncBN model.
Fiend1213 commented on issue #13976: Problem of exporting FP16 SyncBN model.
URL:
https://github.com/apache/incubator-mxnet/issues/13976#issuecomment-457019882
## Environment info (Required)
```
--Python Info--
Version : 3.5.2
Compiler : GCC 5.4.0 20160609
Build: ('default', 'Nov 12 2018 13:43:14')
Arch : ('64bit', 'ELF')
Pip Info---
Version : 9.0.1
Directory: /home/ubuntu/.local/lib/python3.5/site-packages/pip
--MXNet Info---
No MXNet installed.
--System Info--
Platform : Linux-4.4.0-1074-aws-x86_64-with-Ubuntu-16.04-xenial
system : Linux
node : ip-172-31-28-120
release : 4.4.0-1074-aws
version : #84-Ubuntu SMP Thu Dec 6 08:57:58 UTC 2018
--Hardware Info--
machine : x86_64
processor: x86_64
Architecture: x86_64
CPU op-mode(s):32-bit, 64-bit
Byte Order:Little Endian
CPU(s):64
On-line CPU(s) list: 0-63
Thread(s) per core:2
Core(s) per socket:16
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family:6
Model: 79
Model name:Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
Stepping: 1
CPU MHz: 1200.671
CPU max MHz: 3000.
CPU min MHz: 1200.
BogoMIPS: 4600.16
Hypervisor vendor: Xen
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 46080K
NUMA node0 CPU(s): 0-15,32-47
NUMA node1 CPU(s): 16-31,48-63
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm
constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc aperfmperf pni
pclmulqdq monitor est ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt
tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm
3dnowprefetch invpcid_single kaiser fsgsbase bmi1 hle avx2 smep bmi2 erms
invpcid rtm rdseed adx xsaveopt ida
--Network Test--
Setting timeout: 10
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.5594 sec, LOAD:
0.1404 sec.
Timing for FashionMNIST:
https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz,
DNS: 0.0125 sec, LOAD: 0.1469 sec.
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0011
sec, LOAD: 0.3971 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0024 sec, LOAD:
0.3090 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.1254 sec, LOAD:
0.1582 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0090 sec,
LOAD: 0.3155 sec.
```
## Build info (Required if built from source)
```
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
echo "USE_CUDA = 1" >> ./config.mk
echo "USE_CUDA_PATH = /usr/local/cuda" >> ./config.mk
echo "USE_CUDNN = 1" >> ./config.mk
make -j $(nproc)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] aaronmarkham commented on issue #13713: MXNet usage on non-AWS cloud providers
aaronmarkham commented on issue #13713: MXNet usage on non-AWS cloud providers URL: https://github.com/apache/incubator-mxnet/issues/13713#issuecomment-457017132 Assuming we had the content, I think it link from the Ecosystem page... create a new section there. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] samskalicky commented on issue #13976: Problem of exporting FP16 SyncBN model.
samskalicky commented on issue #13976: Problem of exporting FP16 SyncBN model. URL: https://github.com/apache/incubator-mxnet/issues/13976#issuecomment-457016866 Hi @Fiend1213, Can you provide the rest of the info from the issue template to help us debug? ## Environment info (Required) ``` What to do: 1. Download the diagnosis script from https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py 2. Run the script using `python diagnose.py` and paste its output here. ``` Package used (Python/R/Scala/Julia): ## Build info (Required if built from source) Compiler (gcc/clang/mingw/visual studio): MXNet commit hash: (Paste the output of `git rev-parse HEAD` here.) Build config: (Paste the content of config.mk, or the build command.) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham commented on issue #12518: Documentation indices for Random Samplers incomplete
aaronmarkham commented on issue #12518: Documentation indices for Random Samplers incomplete URL: https://github.com/apache/incubator-mxnet/issues/12518#issuecomment-457016725 No, I don't think it has. I can't see any updates for it. These are also missing on beta.mxnet.io. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on a change in pull request #12690: [MXNET-1000] get Ndarray real value and form it from a NDArray
piyushghai commented on a change in pull request #12690: [MXNET-1000] get
Ndarray real value and form it from a NDArray
URL: https://github.com/apache/incubator-mxnet/pull/12690#discussion_r250423216
##
File path:
scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
##
@@ -85,6 +89,47 @@ class NDArraySuite extends FunSuite with BeforeAndAfterAll
with Matchers {
assert(ndarray.toArray === Array(1f, 2f, 3f, 4f))
}
+ test("create NDArray based on Java Matrix") {
+def arrayGen(num : Any) : Array[Any] = {
+ val arrayBuf = num match {
+case f: Float =>
+ val arr = ArrayBuffer[Array[Float]]()
+ for (_ <- 0 until 100) arr += Array(1.0f, 1.0f, 1.0f, 1.0f)
+ arr
+case d: Double =>
+ val arr = ArrayBuffer[Array[Double]]()
+ for (_ <- 0 until 100) arr += Array(1.0d, 1.0d, 1.0d, 1.0d)
+ arr
+case _ => throw new IllegalArgumentException(s"Unsupported Type
${num.getClass}")
+ }
+ Array(
+Array(
+ arrayBuf.toArray
+),
+Array(
+ arrayBuf.toArray
+)
+ )
+}
+val floatData = 1.0f
+var nd = NDArray.toNDArray(arrayGen(floatData))
+require(nd.shape == Shape(2, 1, 100, 4))
+val arr2 = Array(1.0f, 1.0f, 1.0f, 1.0f)
+nd = NDArray.toNDArray(arr2)
+require(nd.shape == Shape(4))
+val doubleData = 1.0d
+nd = NDArray.toNDArray(arrayGen(doubleData))
+require(nd.shape == Shape(2, 1, 100, 4))
+require(nd.dtype == DType.Float64)
+ }
+
+ test("test Visualize") {
+var nd = NDArray.ones(Shape(1, 2, 1000, 1))
+require(nd.toString.split("\n").length == 33)
Review comment:
Should we compare the string representations as well ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] piyushghai commented on a change in pull request #12690: [MXNET-1000] get Ndarray real value and form it from a NDArray
piyushghai commented on a change in pull request #12690: [MXNET-1000] get
Ndarray real value and form it from a NDArray
URL: https://github.com/apache/incubator-mxnet/pull/12690#discussion_r250422997
##
File path: scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
##
@@ -763,6 +823,56 @@ class NDArray private[mxnet](private[mxnet] val handle:
NDArrayHandle,
checkCall(_LIB.mxFloat64NDArraySyncCopyFromCPU(handle, source,
source.length))
}
+ /**
+* Visualize the internal structure of NDArray
+* @return String that show the structure
+*/
+ override def toString: String = {
+val abstractND = buildStringHelper(this, this.shape.length)
+val otherInfo = s""
+s"$abstractND\n$otherInfo"
+ }
+
+ /**
+* Helper function to create formatted NDArray output
+* The NDArray will be represented in a reduced version if too large
+* @param nd NDArray as the input
+* @param totalSpace totalSpace of the lowest dimension
+* @return String format of NDArray
+*/
+ private def buildStringHelper(nd : NDArray, totalSpace : Int) : String = {
+var result = ""
+val THRESHOLD = 10 // longest NDArray[NDArray[...]] to show in full
Review comment:
Can this be MAX_SIZE_THRESHOLD ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] safrooze opened a new pull request #13978: Fixing the doc example for symbolic version of rand_zipfian
safrooze opened a new pull request #13978: Fixing the doc example for symbolic version of rand_zipfian URL: https://github.com/apache/incubator-mxnet/pull/13978 ## Description ## Fixes #12779 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piyushghai commented on a change in pull request #12690: [MXNET-1000] get Ndarray real value and form it from a NDArray
piyushghai commented on a change in pull request #12690: [MXNET-1000] get
Ndarray real value and form it from a NDArray
URL: https://github.com/apache/incubator-mxnet/pull/12690#discussion_r250422657
##
File path: scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
##
@@ -509,6 +510,63 @@ object NDArray extends NDArrayBase {
array(sourceArr, shape, null)
}
+ /**
+* Create a new NDArray based on the structure of source Array
+* @param sourceArr Array[Array...Array[MX_PRIMITIVE_TYPE]...]
+* @param ctx context like to pass in
+* @return an NDArray with the same shape of the input
+*/
+ def toNDArray(sourceArr: Array[_], ctx : Context = null) : NDArray = {
+val shape = ArrayBuffer[Int]()
+shapeGetter(sourceArr, shape, 0)
+val container = new Array[Any](shape.product)
+arrayCombiner(sourceArr, container, 0, container.length - 1)
+val finalArr = container(0) match {
+ case f: Float => array(container.map(_.asInstanceOf[Float]),
Shape(shape), ctx)
+ case d: Double => array(container.map(_.asInstanceOf[Double]),
Shape(shape), ctx)
+ case _ => throw new IllegalArgumentException(s"Unsupported type
${container(0).getClass}")
Review comment:
Should the exception message indicate that the supported types are
Float/Double ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] piyushghai commented on a change in pull request #12690: [MXNET-1000] get Ndarray real value and form it from a NDArray
piyushghai commented on a change in pull request #12690: [MXNET-1000] get
Ndarray real value and form it from a NDArray
URL: https://github.com/apache/incubator-mxnet/pull/12690#discussion_r250421894
##
File path: scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
##
@@ -509,6 +510,63 @@ object NDArray extends NDArrayBase {
array(sourceArr, shape, null)
}
+ /**
+* Create a new NDArray based on the structure of source Array
+* @param sourceArr Array[Array...Array[MX_PRIMITIVE_TYPE]...]
+* @param ctx context like to pass in
+* @return an NDArray with the same shape of the input
+*/
+ def toNDArray(sourceArr: Array[_], ctx : Context = null) : NDArray = {
+val shape = ArrayBuffer[Int]()
+shapeGetter(sourceArr, shape, 0)
+val container = new Array[Any](shape.product)
+arrayCombiner(sourceArr, container, 0, container.length - 1)
+val finalArr = container(0) match {
+ case f: Float => array(container.map(_.asInstanceOf[Float]),
Shape(shape), ctx)
+ case d: Double => array(container.map(_.asInstanceOf[Double]),
Shape(shape), ctx)
+ case _ => throw new IllegalArgumentException(s"Unsupported type
${container(0).getClass}")
Review comment:
Should we add to the method docs as well that it will throw in an
IllegalArgumentException ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] piyushghai commented on a change in pull request #12690: [MXNET-1000] get Ndarray real value and form it from a NDArray
piyushghai commented on a change in pull request #12690: [MXNET-1000] get
Ndarray real value and form it from a NDArray
URL: https://github.com/apache/incubator-mxnet/pull/12690#discussion_r250421577
##
File path:
scala-package/core/src/main/scala/org/apache/mxnet/MX_PRIMITIVES.scala
##
@@ -82,4 +82,10 @@ object MX_PRIMITIVES {
implicit def MX_DoubleToDouble(d: MX_Double) : Double = d.data
+ def isValidType(num : Any) : Boolean = {
Review comment:
Can this be renamed to ```isValidMxPrimitiveType```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] piyushghai opened a new pull request #13977: [MXNET-1293] Adding Iterables instead of List to method signature for infer APIs in Java
piyushghai opened a new pull request #13977: [MXNET-1293] Adding Iterables instead of List to method signature for infer APIs in Java URL: https://github.com/apache/incubator-mxnet/pull/13977 ## Description ## Adding Iterables instead of List to method signature for infer APIs in Java to expand the input types they can take in. This change is backwards compatible as List is a subclass implementing the Iterables interface. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [x] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [x] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change @lanking520 @andrewfayres @zachgk This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vishaalkapoor commented on issue #8715: Documentation for MultiBoxPrior is not substantial
vishaalkapoor commented on issue #8715: Documentation for MultiBoxPrior is not substantial URL: https://github.com/apache/incubator-mxnet/issues/8715#issuecomment-457012297 @piiswrong does https://github.com/apache/incubator-mxnet/pull/5922/files cause '=_Null' in the docs? Does _repr_ need to be '_Null' rather than 'None'? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #7973: C++ demo memory release problem
ThomasDelteil commented on issue #7973: C++ demo memory release problem URL: https://github.com/apache/incubator-mxnet/issues/7973#issuecomment-457012180 @leleamol is it still an issue? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #10395: Android build documentation
ThomasDelteil closed issue #10395: Android build documentation URL: https://github.com/apache/incubator-mxnet/issues/10395 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #10395: Android build documentation
ThomasDelteil commented on issue #10395: Android build documentation URL: https://github.com/apache/incubator-mxnet/issues/10395#issuecomment-457010646 There is also this, https://mxnet.incubator.apache.org/versions/master/faq/smart_device.html You can also try the arm build with the java bindings @larroy can maybe shed more light This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #10937: c++ API, fixed batch size for predictor.
ThomasDelteil commented on issue #10937: c++ API, fixed batch size for predictor. URL: https://github.com/apache/incubator-mxnet/issues/10937#issuecomment-457010136 @leleamol could you pick this up ? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Ishitori commented on issue #10395: Android build documentation
Ishitori commented on issue #10395: Android build documentation URL: https://github.com/apache/incubator-mxnet/issues/10395#issuecomment-457009968 And the second one is here: https://github.com/Leliana/WhatsThis This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #10003: C++ API Documentation and Tutorial
ThomasDelteil closed issue #10003: C++ API Documentation and Tutorial URL: https://github.com/apache/incubator-mxnet/issues/10003 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sad- commented on issue #12576: Can anybody provide a kv-store (distributed training) example with cpp package?
sad- commented on issue #12576: Can anybody provide a kv-store (distributed training) example with cpp package? URL: https://github.com/apache/incubator-mxnet/issues/12576#issuecomment-457009869 @xiaolin-cheng were you able to implement what you needed with the generated c++ documentation? if you were, you can contribute your code as an example for distributed training with c++ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] thomelane commented on issue #10395: Android build documentation
thomelane commented on issue #10395: Android build documentation URL: https://github.com/apache/incubator-mxnet/issues/10395#issuecomment-457009886 One option for deploying MXNet models is via TVM. See https://docs.tvm.ai/deploy/android.html This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vishaalkapoor commented on issue #11100: Installation instructions for Windows don't work
vishaalkapoor commented on issue #11100: Installation instructions for Windows don't work URL: https://github.com/apache/incubator-mxnet/issues/11100#issuecomment-457009208 What are the outstanding issues for this issue? Can we have a checklist so that it is easier to keep track of progress? Adding a few names for visibility: @lebeg @aaronmarkham @piyushghai This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] NRauschmayr commented on issue #10518: Does MXNet have official WaveNet model
NRauschmayr commented on issue #10518: Does MXNet have official WaveNet model URL: https://github.com/apache/incubator-mxnet/issues/10518#issuecomment-457008169 Here is a work-in progress example https://github.com/seujung/WaveNet-gluon This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #12393: Deadlock in save_checkpoint when using threading
ThomasDelteil closed issue #12393: Deadlock in save_checkpoint when using threading URL: https://github.com/apache/incubator-mxnet/issues/12393 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sad- commented on issue #8806: Not all LSTM layout options are specified
sad- commented on issue #8806: Not all LSTM layout options are specified URL: https://github.com/apache/incubator-mxnet/issues/8806#issuecomment-457006698 The assertion error message gives the current possible valid layouts so not really a doc issue. If the 'NTC' layout feature or other layout is implemented then doc will be updated alongside. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Ishitori commented on issue #12393: Deadlock in save_checkpoint when using threading
Ishitori commented on issue #12393: Deadlock in save_checkpoint when using threading URL: https://github.com/apache/incubator-mxnet/issues/12393#issuecomment-457006123 Unfortunately, overall MXNet is not thread safe. You can see why in this part of the documentation: https://mxnet.incubator.apache.org/versions/master/architecture/overview.html?#push-and-wait. If it answers your question, please close the ticket. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] safrooze commented on issue #13227: Missing return value documentation for nd.random.* and sym.random.*
safrooze commented on issue #13227: Missing return value documentation for nd.random.* and sym.random.* URL: https://github.com/apache/incubator-mxnet/issues/13227#issuecomment-457006150 @zboldyga gentle check to see if you've submitted a PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mxnet-label-bot commented on issue #13976: Problem of exporting FP16 SyncBN model.
mxnet-label-bot commented on issue #13976: Problem of exporting FP16 SyncBN model. URL: https://github.com/apache/incubator-mxnet/issues/13976#issuecomment-457005276 Hey, this is the MXNet Label Bot. Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it. Here are my recommended labels: Performance This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Fiend1213 opened a new issue #13976: Problem of exporting FP16 SyncBN model.
Fiend1213 opened a new issue #13976: Problem of exporting FP16 SyncBN model.
URL: https://github.com/apache/incubator-mxnet/issues/13976
## Description
This is a problem when exporting fp16 model containing SyncBN.
```
import mxnet as mx
mx.random.seed(42)
def data_xform(data):
"""Move channel axis to the beginning, cast to float32, and normalize to
[0, 1]."""
return mx.nd.moveaxis(data, 2, 0).astype('float32') / 255
train_data =
mx.gluon.data.vision.MNIST(train=True).transform_first(data_xform)
val_data =
mx.gluon.data.vision.MNIST(train=False).transform_first(data_xform)
batch_size = 2000
train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True,
batch_size=batch_size)
val_loader = mx.gluon.data.DataLoader(val_data, shuffle=False,
batch_size=batch_size)
net = mx.gluon.nn.HybridSequential()
with net.name_scope():
net.add(mx.gluon.nn.Conv2D(64, (3, 3)))
net.add(mx.gluon.contrib.nn.SyncBatchNorm())
net.add(mx.gluon.nn.Conv2D(64, (3, 3)))
net.add(mx.gluon.contrib.nn.SyncBatchNorm())
net.add(mx.gluon.nn.Dense(128))
net.add(mx.gluon.nn.Dense(10))
net.hybridize()
net.cast('float16')
print('finish build the network')
ctx = [mx.gpu(int(id)) for id in [0,1,2,3,4,5,6,7]]
net.initialize(mx.init.Normal(0.01), ctx=ctx)
print('finish initializing')
trainer = mx.gluon.Trainer(
params=net.collect_params(),
optimizer='sgd',
optimizer_params={'learning_rate': 0.04, 'multi_precision':True},
)
loss_function = mx.gluon.loss.SoftmaxCrossEntropyLoss()
metric = mx.metric.Accuracy()
num_epochs = 10
for epoch in range(num_epochs):
print('start training')
for inputs, labels in train_loader:
data = mx.gluon.utils.split_and_load(inputs, ctx_list=ctx,
batch_axis=0)
label = mx.gluon.utils.split_and_load(labels, ctx_list=ctx,
batch_axis=0)
losses = []
with mx.autograd.record():
for X, Y in zip(data, label):
outputs = net(X.astype('float16'))
loss = loss_function(outputs.astype('float32'), Y)
losses.append(loss)
for l in losses:
l.backward()
trainer.step(batch_size=inputs.shape[0])
net.export('mnist_syncbn_fp16.params')
```
## Error Message:
```
File "/home/ubuntu/Workspace/incubator-mxnet/python/mxnet/gluon/block.py",
line 900, in export
ndarray.save('%s-%04d.params'%(path, epoch), arg_dict)
File
"/home/ubuntu/Workspace/incubator-mxnet/python/mxnet/ndarray/utils.py", line
273, in save
keys))
File "/home/ubuntu/Workspace/incubator-mxnet/python/mxnet/base.py", line
255, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [18:43:24] include/mxnet/././tensor_blob.h:203: Check
failed: mshadow::DataType::kFlag == type_flag_ TBlob.get_with_shape:
data type do not match specified type.Expected: 2 v.s. given 0
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] thomelane commented on issue #12828: Navigation for Docs Broken
thomelane commented on issue #12828: Navigation for Docs Broken URL: https://github.com/apache/incubator-mxnet/issues/12828#issuecomment-457005025 @aaronmarkham reckon we could just add links to the headers in the markdown `docs/api/python/index.md`? Since the headers are getting added to the top level of the navigation tree. So `## Autograd API` -> `## [Autograd API](autograd/autograd.md)` Also, for Autograd and Contrib, there's nothing underneath so wonder if we wanted to avoid generating the tree at all, and just use headers? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #10064: Explaining `need_top_grad` and `declare_backward_dependency` in the documentation of CustomOp
ThomasDelteil commented on issue #10064: Explaining `need_top_grad` and `declare_backward_dependency` in the documentation of CustomOp URL: https://github.com/apache/incubator-mxnet/issues/10064#issuecomment-457004698 @sxjscience any update on that? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #10409: https://newdocs.readthedocs.io is outdated
ThomasDelteil closed issue #10409: https://newdocs.readthedocs.io is outdated URL: https://github.com/apache/incubator-mxnet/issues/10409 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #11867: we need to update the doc of scatter_nd and gather_nd
ThomasDelteil closed issue #11867: we need to update the doc of scatter_nd and gather_nd URL: https://github.com/apache/incubator-mxnet/issues/11867 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #11867: we need to update the doc of scatter_nd and gather_nd
ThomasDelteil commented on issue #11867: we need to update the doc of scatter_nd and gather_nd URL: https://github.com/apache/incubator-mxnet/issues/11867#issuecomment-457003589 closing for now, please reopen with details of what needs to be updated if necessary, thanks This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] safrooze commented on issue #11557: Logging in Mxnet, with different level (verbose, warning, error)
safrooze commented on issue #11557: Logging in Mxnet, with different level (verbose, warning, error) URL: https://github.com/apache/incubator-mxnet/issues/11557#issuecomment-457002895 @dmidge8 one option for managing logs when they're all enabled would be to use a log viewer like [glogg](https://glogg.bonnefon.org/). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sad- commented on issue #11867: we need to update the doc of scatter_nd and gather_nd
sad- commented on issue #11867: we need to update the doc of scatter_nd and gather_nd URL: https://github.com/apache/incubator-mxnet/issues/11867#issuecomment-457002260 echoing @piyushghai are there still pending updates for this? @sandeep-krishnamurthy @haojin2 can this be closed now? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil opened a new pull request #13975: Fix quote on LBSGD docs
ThomasDelteil opened a new pull request #13975: Fix quote on LBSGD docs URL: https://github.com/apache/incubator-mxnet/pull/13975 ## Description ## fix https://github.com/apache/incubator-mxnet/issues/12171 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin opened a new issue #12171: LBSGD doc not rendering correctly
eric-haibin-lin opened a new issue #12171: LBSGD doc not rendering correctly URL: https://github.com/apache/incubator-mxnet/issues/12171 https://user-images.githubusercontent.com/5545640/44124041-07d4bf4a-9fe0-11e8-8463-f963e3e4af83.png;> This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] thomelane commented on issue #13301: building 1.2.0 tag fails with tvm error
thomelane commented on issue #13301: building 1.2.0 tag fails with tvm error URL: https://github.com/apache/incubator-mxnet/issues/13301#issuecomment-457000664 Was going to suggest updating the submodules, but just spotted you've already done that. You did something like this? ``` git checkout d283c516bc97fa7c1ed18d2e32d28812b473979f git submodule update --recursive --init ``` @yzhliu couldn't find any reference to `tvm/runtime/packed_func.h` in the past for tvm. any idea where it came from? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] zachgk opened a new pull request #13974: Fix flaky maven binary download
zachgk opened a new pull request #13974: Fix flaky maven binary download URL: https://github.com/apache/incubator-mxnet/pull/13974 ## Description ## This PR adds a second mirror when downloading the maven binary as the first might show flaky behavior (http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-13966/1/pipeline). Additionally, it adds the "set -ex" to the ubuntu publish install script to avoid silent failures. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [x] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change @marcoabreu @lanking520 @piyushghai @andrewfayres This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil closed issue #12171: LBSGD doc not rendering correctly
ThomasDelteil closed issue #12171: LBSGD doc not rendering correctly URL: https://github.com/apache/incubator-mxnet/issues/12171 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sad- commented on issue #12518: Documentation indices for Random Samplers incomplete
sad- commented on issue #12518: Documentation indices for Random Samplers incomplete URL: https://github.com/apache/incubator-mxnet/issues/12518#issuecomment-456998657 @aaronmarkham has the fix for this gone out yet? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil commented on issue #13580: [Clojure] - Add Clojure to the main README
ThomasDelteil commented on issue #13580: [Clojure] - Add Clojure to the main README URL: https://github.com/apache/incubator-mxnet/issues/13580#issuecomment-456995938 See https://github.com/dmlc/web-data/pull/152 and https://github.com/apache/incubator-mxnet/pull/13973 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThomasDelteil opened a new pull request #13973: Update README.md to include new bindings
ThomasDelteil opened a new pull request #13973: Update README.md to include new bindings URL: https://github.com/apache/incubator-mxnet/pull/13973 ## Description ## Simply updating the readme to add the new bindings supports This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] apeforest commented on issue #11203: ConvTranspose2d giving incorrect output
apeforest commented on issue #11203: ConvTranspose2d giving incorrect output URL: https://github.com/apache/incubator-mxnet/issues/11203#issuecomment-456994832 Confirmed that the bug is in `unpack_patch2col` and `pack_col2patch` when dilation is specified. Replacing them with `col2im` and `im2col` resolved the bug. PR coming out soon. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] thomelane commented on issue #8234: A bug in VGG_FC_ILSVRC_16_layers-symbol.json
thomelane commented on issue #8234: A bug in VGG_FC_ILSVRC_16_layers-symbol.json URL: https://github.com/apache/incubator-mxnet/issues/8234#issuecomment-456991703 @tornadomeet are you the owner of the original baidu and dropbox hosting? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] safrooze commented on issue #13713: MXNet usage on non-AWS cloud providers
safrooze commented on issue #13713: MXNet usage on non-AWS cloud providers URL: https://github.com/apache/incubator-mxnet/issues/13713#issuecomment-456990024 @aaronmarkham Have you looked at this issue to see where it would fit on the website? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wkcn commented on issue #13928: MXNet 1.5.0 is slower than 1.3.0 when intputs are variant
wkcn commented on issue #13928: MXNet 1.5.0 is slower than 1.3.0 when intputs are variant URL: https://github.com/apache/incubator-mxnet/issues/13928#issuecomment-456988184 @zhreshold solved. Thank you！ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wkcn closed issue #13928: MXNet 1.5.0 is slower than 1.3.0 when intputs are variant
wkcn closed issue #13928: MXNet 1.5.0 is slower than 1.3.0 when intputs are variant URL: https://github.com/apache/incubator-mxnet/issues/13928 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Static build for Python (#13916)
This is an automated email from the ASF dual-hosted git repository.
marcoabreu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new f033aec Static build for Python (#13916)
f033aec is described below
commit f033aec603c75547f6abcb95d779650a0e1b5025
Author: Lanking
AuthorDate: Wed Jan 23 14:11:23 2019 -0800
Static build for Python (#13916)
* add python unit test
* address comments
* switch sanity test to Gluon module test
* We don't run tests (╯‵□′)╯︵┻━┻
* add variant in the environment variable
* add document improvement
* kill the conflict
---
ci/docker/runtime_functions.sh | 8
ci/jenkins/Jenkins_steps.groovy | 13 +
ci/jenkins/Jenkinsfile_unix_cpu | 1 +
ci/publish/README.md | 16
ci/publish/python/build.sh | 26 ++
tools/staticbuild/README.md | 10 +-
tools/staticbuild/build_wheel.sh | 5 +
7 files changed, 62 insertions(+), 17 deletions(-)
diff --git a/ci/docker/runtime_functions.sh b/ci/docker/runtime_functions.sh
index 9f9dd0c..f763137 100755
--- a/ci/docker/runtime_functions.sh
+++ b/ci/docker/runtime_functions.sh
@@ -1295,6 +1295,14 @@ build_scala_static_mkl() {
popd
}
+build_static_python_mkl() {
+set -ex
+pushd .
+export mxnet_variant=mkl
+./ci/publish/python/build.sh
+popd
+}
+
publish_scala_build() {
set -ex
pushd .
diff --git a/ci/jenkins/Jenkins_steps.groovy b/ci/jenkins/Jenkins_steps.groovy
index 984672c..f1703ba 100644
--- a/ci/jenkins/Jenkins_steps.groovy
+++ b/ci/jenkins/Jenkins_steps.groovy
@@ -534,6 +534,19 @@ def test_static_scala_cpu() {
}]
}
+def test_static_python_cpu() {
+ return ['Static build CPU 14.04 Python' : {
+node(NODE_LINUX_CPU) {
+ws('workspace/ut-publish-python-cpu') {
+ timeout(time: max_time, unit: 'MINUTES') {
+utils.init_git()
+utils.docker_run("publish.ubuntu1404_cpu",
'build_static_python_mkl', false)
+ }
+}
+}
+ }]
+}
+
def test_unix_python2_cpu() {
return ['Python2: CPU': {
node(NODE_LINUX_CPU) {
diff --git a/ci/jenkins/Jenkinsfile_unix_cpu b/ci/jenkins/Jenkinsfile_unix_cpu
index 9446348..234a65b 100644
--- a/ci/jenkins/Jenkinsfile_unix_cpu
+++ b/ci/jenkins/Jenkinsfile_unix_cpu
@@ -60,6 +60,7 @@ core_logic: {
custom_steps.test_unix_onnx_cpu(),
custom_steps.test_unix_cpp_cpu(),
custom_steps.test_static_scala_cpu(),
+custom_steps.test_static_python_cpu(),
/* Disabled due to master build failure:
*
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/master/1221/pipeline/
* https://github.com/apache/incubator-mxnet/issues/11801
diff --git a/ci/publish/README.md b/ci/publish/README.md
index 2344cdf..f1ece6f 100644
--- a/ci/publish/README.md
+++ b/ci/publish/README.md
@@ -1,9 +1,9 @@
# MXNet Publish Settings
-This folder contains the configuration of restricted node on Jenkins for the
publish. It also contains a folder called `scala` that contains everything
required for scala publish. In this `README`, we would bring a brief
walkthrough of the Jenkins configuration as well as the usages of the scala
deployment files.
+This folder contains the configuration for restricted nodes on Jenkins for the
publishing MXNet artifacts. It also contains a folder called `scala` that
contains everything required for publishing to Maven. In this `README`, we
provide a brief walkthrough of the Jenkins configuration as well as the usage
of the Scala deployment files. Python publishing is TBD.
## Jenkins
-Currently, Jenkins contains three build stages, namely `Build Packages`, `Test
Packages` and `Deploy Packages`. During the `build package` stages, all
dependencies will be built and a Scala package would be created. In the second
stage, the package created from the previous stage would move to this stage to
specifically run the tests. In the final stage, the packages passed the test
would be deployed by the instances.
+Currently, Jenkins contains three build stages, namely `Build Packages`, `Test
Packages` and `Deploy Packages`. During the `build package` stages, all
dependencies are built and a Scala package are created. In the second stage,
the package created from the previous stage moves to this stage to specifically
run the tests. In the final stage, the packages that pass the tests are
deployed by the instances.
The job is scheduled to be triggered every 24 hours on a [restricted
instance](http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/restricted-publish-artifacts).
@@ -17,11 +17,11 @@ All packages are currently built in `Ubuntu 14.04`. All
Dockerfile used for publ
Apart from that, the script used to create the environment and
[GitHub] marcoabreu merged pull request #13916: Static build for Python
marcoabreu merged pull request #13916: Static build for Python URL: https://github.com/apache/incubator-mxnet/pull/13916 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu commented on issue #13916: Static build for Python
marcoabreu commented on issue #13916: Static build for Python URL: https://github.com/apache/incubator-mxnet/pull/13916#issuecomment-456986596 Looks great, thanks a ton! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham commented on a change in pull request #13963: Fix website error pages
aaronmarkham commented on a change in pull request #13963: Fix website error
pages
URL: https://github.com/apache/incubator-mxnet/pull/13963#discussion_r250392567
##
File path: docs/build_version_doc/artifacts/404.html
##
@@ -0,0 +1,50 @@
+
+
+
+
+
+
+
+{ margin: 0; padding: 0; }
+
+html {
+ background:
url('https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet-background-compressed.jpeg')
no-repeat center center fixed;
+ -webkit-background-size: cover;
+ -moz-background-size: cover;
+ -o-background-size: cover;
+ background-size: cover;
+}
+
+h1, p {
+ color: white;
+ font-family: verdana;
+}
+
+a:link {
+ color: white;
+}
+
+a:visited {
+ color: linen;
+}
+
+a:hover {
+ color: powderblue;
+}
+
+a:active {
+ color: aqua;
+}
+
+
+
+
+
+ https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet_logo.png;>
+
+
+Page Does Not Exist
+If you’re here that means you requested a page that doesn’t exist.
Sorry about that! Maybe try the search box to find what you’re looking for, or
navigate to the https://mxnet.incubator.apache.org/index.html;>Home
Page. Also, make sure you’re looking in the correct version, as some
features may only be available in https://github.com/apache/incubator-mxnet/releases;>newer versions or
the https://mxnet.incubator.apache.org/versions/master;>master
branch.
Review comment:
Yes, I fixed that. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] marcoabreu commented on issue #13952: Fix MXNet R package build
marcoabreu commented on issue #13952: Fix MXNet R package build URL: https://github.com/apache/incubator-mxnet/pull/13952#issuecomment-456985929 Ideally, we could do some smoketests for different compilation options by running a simple test to ensure everything is working. Considering we test different backends in Python already, I don't think there's much benefit of now testing the full spectrum across the different frontend languages. As far as I understand, the goal is to ensure everything gets compiled and linked properly, right? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu commented on a change in pull request #13952: Fix MXNet R package build
marcoabreu commented on a change in pull request #13952: Fix MXNet R package
build
URL: https://github.com/apache/incubator-mxnet/pull/13952#discussion_r250391298
##
File path: ci/jenkins/Jenkinsfile_unix_gpu
##
@@ -53,6 +53,7 @@ core_logic: {
custom_steps.test_unix_python3_tensorrt_gpu(),
custom_steps.test_unix_perl_gpu(),
custom_steps.test_unix_r_gpu(),
+custom_steps.test_unix_r_mkldnn_gpu(),
Review comment:
I'm not sure whether we need GPU+MKLDNN for R since we already test it in
python. Could you elaborate what concerns you are specifically having?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io removed a comment on issue #13966: upload example code for snail with gluon
codecov-io removed a comment on issue #13966: upload example code for snail with gluon URL: https://github.com/apache/incubator-mxnet/pull/13966#issuecomment-456860141 # [Codecov](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=h1) Report > Merging [#13966](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-mxnet/commit/4c88f3049695b9c751a6d1a228853f18b2733358?src=pr=desc) will **increase** coverage by `0.34%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=tree) ```diff @@Coverage Diff @@ ## master #13966 +/- ## == + Coverage 78.46%78.8% +0.34% == Files 763 767 +4 Lines 8407784546 +469 Branches 3231 3231 == + Hits6596966629 +660 + Misses 1724017049 -191 Partials 868 868 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=tree) | Coverage Δ | | |---|---|---| | [cpp-package/include/mxnet-cpp/initializer.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-Y3BwLXBhY2thZ2UvaW5jbHVkZS9teG5ldC1jcHAvaW5pdGlhbGl6ZXIuaA==) | `28.35% <0%> (-71.65%)` | :arrow_down: | | [cpp-package/include/mxnet-cpp/shape.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-Y3BwLXBhY2thZ2UvaW5jbHVkZS9teG5ldC1jcHAvc2hhcGUuaA==) | `78.46% <0%> (-21.54%)` | :arrow_down: | | [cpp-package/include/mxnet-cpp/symbol.hpp](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-Y3BwLXBhY2thZ2UvaW5jbHVkZS9teG5ldC1jcHAvc3ltYm9sLmhwcA==) | `96.22% <0%> (-3.78%)` | :arrow_down: | | [src/operator/nn/depthwise\_convolution\_tf.cuh](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-c3JjL29wZXJhdG9yL25uL2RlcHRod2lzZV9jb252b2x1dGlvbl90Zi5jdWg=) | `84.7% <0%> (-1.18%)` | :arrow_down: | | [src/operator/softmax\_output-inl.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-c3JjL29wZXJhdG9yL3NvZnRtYXhfb3V0cHV0LWlubC5o) | `59.82% <0%> (-0.88%)` | :arrow_down: | | [src/operator/mxnet\_op.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-c3JjL29wZXJhdG9yL214bmV0X29wLmg=) | `80.55% <0%> (-0.76%)` | :arrow_down: | | [src/operator/contrib/dgl\_graph.cc](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-c3JjL29wZXJhdG9yL2NvbnRyaWIvZGdsX2dyYXBoLmNj) | `91.77% <0%> (-0.12%)` | :arrow_down: | | [cpp-package/include/mxnet-cpp/operator.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-Y3BwLXBhY2thZ2UvaW5jbHVkZS9teG5ldC1jcHAvb3BlcmF0b3IuaA==) | `100% <0%> (ø)` | :arrow_up: | | [cpp-package/include/mxnet-cpp/ndarray.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-Y3BwLXBhY2thZ2UvaW5jbHVkZS9teG5ldC1jcHAvbmRhcnJheS5o) | `100% <0%> (ø)` | :arrow_up: | | [cpp-package/include/mxnet-cpp/op\_suppl.h](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree#diff-Y3BwLXBhY2thZ2UvaW5jbHVkZS9teG5ldC1jcHAvb3Bfc3VwcGwuaA==) | `100% <0%> (ø)` | | | ... and [31 more](https://codecov.io/gh/apache/incubator-mxnet/pull/13966/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=footer). Last update [4c88f30...e345cd3](https://codecov.io/gh/apache/incubator-mxnet/pull/13966?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lanking520 commented on issue #13916: Static build for Python
lanking520 commented on issue #13916: Static build for Python URL: https://github.com/apache/incubator-mxnet/pull/13916#issuecomment-456975120 @szha @marcoabreu Please check again to see if this meet yours requirement This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Update autoencoder example (#12933)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 402e985 Update autoencoder example (#12933) 402e985 is described below commit 402e985b1482bac6e8b89896bab18024e3a9fcfc Author: Thomas Delteil AuthorDate: Wed Jan 23 13:29:05 2019 -0800 Update autoencoder example (#12933) * Fixing the autoencoder example * adding pointer to VAE * fix typos * Update README.md * Updating notebook * Update after comments * Update README.md * Update README.md * Retrigger build * Updates after review --- example/autoencoder/README.md | 24 +- example/autoencoder/autoencoder.py | 206 .../autoencoder/convolutional_autoencoder.ipynb| 543 + example/autoencoder/data.py| 34 -- example/autoencoder/mnist_sae.py | 100 example/autoencoder/model.py | 78 --- example/autoencoder/solver.py | 151 -- 7 files changed, 557 insertions(+), 579 deletions(-) diff --git a/example/autoencoder/README.md b/example/autoencoder/README.md index 7efa30a..960636c 100644 --- a/example/autoencoder/README.md +++ b/example/autoencoder/README.md @@ -1,16 +1,20 @@ -# Example of Autencoder +# Example of a Convolutional Autoencoder -Autoencoder architecture is often used for unsupervised feature learning. This [link](http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/) contains an introduction tutorial to autoencoders. This example illustrates a simple autoencoder using stack of fully-connected layers for both encoder and decoder. The number of hidden layers and size of each hidden layer can be customized using command line arguments. +Autoencoder architectures are often used for unsupervised feature learning. This [link](http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/) contains an introduction tutorial to autoencoders. This example illustrates a simple autoencoder using a stack of convolutional layers for both the encoder and the decoder. -## Training Stages -This example uses a two-stage training. In the first stage, each layer of encoder and its corresponding decoder are trained separately in a layer-wise training loop. In the second stage the entire autoencoder network is fine-tuned end to end. + + + +([Diagram source](https://towardsdatascience.com/autoencoders-introduction-and-implementation-3f40483b0a85)) + + +The idea of an autoencoder is to learn to use bottleneck architecture to encode the input and then try to decode it to reproduce the original. By doing so, the network learns to effectively compress the information of the input, the resulting embedding representation can then be used in several domains. For example as featurized representation for visual search, or in anomaly detection. ## Dataset -The dataset used in this example is [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. This example uses scikit-learn module to download this dataset. -## Simple autoencoder example -mnist_sae.py: this example uses a simple auto-encoder architecture to encode and decode MNIST images with size of 28x28 pixels. It contains several command line arguments. Pass -h (or --help) to view all available options. To start the training on CPU (use --gpu option for training on GPU) using default options: +The dataset used in this example is [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset. + +## Variational Autoencoder + +You can check an example of variational autoencoder [here](https://gluon.mxnet.io/chapter13_unsupervised-learning/vae-gluon.html) -``` -python mnist_sae.py -``` diff --git a/example/autoencoder/autoencoder.py b/example/autoencoder/autoencoder.py deleted file mode 100644 index 47931e5..000 --- a/example/autoencoder/autoencoder.py +++ /dev/null @@ -1,206 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and
[GitHub] KellenSunderland merged pull request #12933: Update autoencoder example
KellenSunderland merged pull request #12933: Update autoencoder example URL: https://github.com/apache/incubator-mxnet/pull/12933 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lanking520 commented on issue #13559: fix for opencv4
lanking520 commented on issue #13559: fix for opencv4 URL: https://github.com/apache/incubator-mxnet/pull/13559#issuecomment-456970202 Do we still have OpenCV 2 and 3 support if we switch the namespace? Since CI now are using CV2 and publish using CV3 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
