[GitHub] [incubator-singa] dcslin opened a new pull request #548: DNNL(upgrade of MKLDNN) integration and add softmax
dcslin opened a new pull request #548: DNNL(upgrade of MKLDNN) integration and add softmax URL: https://github.com/apache/incubator-singa/pull/548 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #547: SINGA-493 Add an additional build path
chrishkchris opened a new pull request #547: SINGA-493 Add an additional build path URL: https://github.com/apache/incubator-singa/pull/547 Add an webpage build path in build.sh This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin edited a comment on issue #541: added 4d test on batchnorm
dcslin edited a comment on issue #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541#issuecomment-540867039 HI @joddiy , could you share how did you call singa api? the test you mentioned is tested in here https://github.com/apache/incubator-singa/pull/541/files#diff-5e6248de6199ea6f777c73d3ea2643a5R62 ``` python def _np_bn_testing(x, scale, bias, rm, rv, momentum=0.1, e=1e-5): ... return scale * (x - rm) / np.sqrt(rv + e) + bias ``` for the `test_mode`, kindly use this api `GpuBatchNormForwardInference` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #541: added 4d test on batchnorm
dcslin commented on issue #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541#issuecomment-540867039 HI @joddiy , could you share how did you call singa api? the test you mentioned is tested in here https://github.com/apache/incubator-singa/pull/541/files#diff-5e6248de6199ea6f777c73d3ea2643a5R62 for the `test_mode`, kindly use this api `GpuBatchNormForwardInference` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy edited a comment on issue #541: added 4d test on batchnorm
joddiy edited a comment on issue #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541#issuecomment-540600410 hi, shicong, the result still has a little error, please check with the following case: ``` def _batchnorm_test_mode(x, s, bias, mean, var, epsilon=1e-5): # type: ignore dims_x = len(x.shape) dim_ones = (1,) * (dims_x - 2) s = s.reshape(-1, *dim_ones) bias = bias.reshape(-1, *dim_ones) mean = mean.reshape(-1, *dim_ones) var = var.reshape(-1, *dim_ones) return s * (x - mean) / np.sqrt(var + epsilon) + bias # input size: (1, 2, 1, 3) x = np.array(-1, 0, 1]], [[2, 3, 4).astype(np.float32) s = np.array([1.0, 1.5]).astype(np.float32) bias = np.array([0, 1]).astype(np.float32) mean = np.array([0, 3]).astype(np.float32) var = np.array([1, 1.5]).astype(np.float32) y = _batchnorm_test_mode(x, s, bias, mean, var).astype(np.float32) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #541: added 4d test on batchnorm
joddiy commented on issue #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541#issuecomment-540600410 hi, shicong, the result still has a little error, please check with the following case: ``` def _batchnorm_test_mode(x, s, bias, mean, var, epsilon=1e-5): # type: ignore dims_x = len(x.shape) dim_ones = (1,) * (dims_x - 2) s = s.reshape(-1, *dim_ones) bias = bias.reshape(-1, *dim_ones) mean = mean.reshape(-1, *dim_ones) var = var.reshape(-1, *dim_ones) return s * (x - mean) / np.sqrt(var + epsilon) + bias # input size: (1, 2, 1, 3) x = np.array(-1, 0, 1]], [[2, 3, 4).astype(np.float32) s = np.array([1.0, 1.5]).astype(np.float32) bias = np.array([0, 1]).astype(np.float32) mean = np.array([0, 3]).astype(np.float32) var = np.array([1, 1.5]).astype(np.float32) y = _batchnorm_test_mode(x, s, bias, mean, var).astype(np.float32) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #540: added softmax with axis
joddiy commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-540567614 ready to merge This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin closed pull request #538: [WIP] add Cudnn mode as optional param to batch norm and softmax
dcslin closed pull request #538: [WIP] add Cudnn mode as optional param to batch norm and softmax URL: https://github.com/apache/incubator-singa/pull/538 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #538: [WIP] add Cudnn mode as optional param to batch norm and softmax
dcslin commented on issue #538: [WIP] add Cudnn mode as optional param to batch norm and softmax URL: https://github.com/apache/incubator-singa/pull/538#issuecomment-540511768 split to https://github.com/apache/incubator-singa/pull/540 and https://github.com/apache/incubator-singa/pull/541 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa-site] nudles merged pull request #1: SINGA-493 Update SINGA website
nudles merged pull request #1: SINGA-493 Update SINGA website URL: https://github.com/apache/incubator-singa-site/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #546: SINGA-491 Remaining LGTM Python Code Cleaning
nudles merged pull request #546: SINGA-491 Remaining LGTM Python Code Cleaning URL: https://github.com/apache/incubator-singa/pull/546 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #546: SINGA-491 Remaining LGTM Python Code Cleaning
chrishkchris opened a new pull request #546: SINGA-491 Remaining LGTM Python Code Cleaning URL: https://github.com/apache/incubator-singa/pull/546 This is to clean some remaining LGTM Python code alerts that I am capable to clean. This is the last PR for my current code cleaning, because after this PR I think the current code is clean enough. Meanwhile, in the future I can still help to clean when there are new LGTM alerts. For some false positive alarms in LGTM, I have submitted the query in semmle/ql that might help them to debug their system: https://github.com/Semmle/ql/issues https://github.com/Semmle/ql/issues/2101 https://github.com/Semmle/ql/issues/2071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #540: added softmax with axis
dcslin commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-539904174 Hi @joddiy, thank you for the review. kindly find the updated version which fixed the mentioned issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #540: added softmax with axis
dcslin commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-539628035 > > > still has some problems, the output of multiple dimension inputs is not correct. > > > please check: > > > ``` > > > x_0 = np.array([[0, 1, 2, 3], [1, 10001, 10002, 10003]]).astype(np.float32) > > > # axis is 1 > > > # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428], > > > # [0.0320586, 0.08714432, 0.23688284, 0.64391428]] > > > ``` > > > > > > Hi @joddiy , this is updated, could you please help to review? > > Hi shicong, the axis and output are almost correct. However, the result will overflow if the input values are too big. For example, for input of [0, 1, 2, 3], the result is correct. But for input of 1, 10001, 10002, 10003], the result will be [nan, nan, nan, nan]. > > Please use this formation: > > ``` > def softmax(x): > """Compute softmax values for each sets of scores in x.""" > e_x = np.exp(x - np.max(x)) > return e_x / e_x.sum() > ``` > > **not directly use x as input, instead subtracting the max value of x from x to avoid the overflow.** Hi @joddiy , thank you for the comment. I am still looking into this issue, as `max()` is not currently implemented in backend yet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #541: added 4d test on batchnorm
dcslin commented on issue #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541#issuecomment-539449929 HI @joddiy , could you please help to review this test on batchnorm value? which also showing that the required params/output by ONNX are given. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy opened a new pull request #545: SINGA-494 Singa autograd improvement
joddiy opened a new pull request #545: SINGA-494 Singa autograd improvement URL: https://github.com/apache/incubator-singa/pull/545 ## Background: some autograd ops cannot satisfy the onnx demand, as following: ### conv, averagepool, maxpool - only support 2d input, i.e, N/*C/*W/*H - not support SAME_UPPER, SAME_LOWER, count_include_pad and ceil_mode ### reshape - not support zero_dim, zero_and_negative_dim ### concat - not support 1d ### matmul - only support 2d ### min, max - only support 2 inputs ### add - not support broadcast ### and, or, xor - not support broadcast ### div, pow, prelu - not support broadcast This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #540: added softmax with axis
joddiy commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-538984488 > > still has some problems, the output of multiple dimension inputs is not correct. > > please check: > > ``` > > x_0 = np.array([[0, 1, 2, 3], [1, 10001, 10002, 10003]]).astype(np.float32) > > # axis is 1 > > # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428], > > # [0.0320586, 0.08714432, 0.23688284, 0.64391428]] > > ``` > > Hi @joddiy , this is updated, could you please help to review? Hi shicong, the axis and output are almost correct. However, the result will overflow if the input values are too big. For example, for input of [0, 1, 2, 3], the result is correct. But for input of 1, 10001, 10002, 10003], the result will be [nan, nan, nan, nan]. Please use this formation: ``` def softmax(x): """Compute softmax values for each sets of scores in x.""" e_x = np.exp(x - np.max(x)) return e_x / e_x.sum() ``` **not directly use x as input, instead subtracting the max value of x from x to avoid the overflow.** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #540: added softmax with axis
dcslin commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-538955169 > still has some problems, the output of multiple dimension inputs is not correct. > please check: > > ``` > x_0 = np.array([[0, 1, 2, 3], [1, 10001, 10002, 10003]]).astype(np.float32) > # axis is 1 > # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428], > # [0.0320586, 0.08714432, 0.23688284, 0.64391428]] > ``` Hi @joddiy , this is updated, could you please help to review? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa-site] chrishkchris opened a new pull request #1: SINGA-493 Update SINGA website
chrishkchris opened a new pull request #1: SINGA-493 Update SINGA website URL: https://github.com/apache/incubator-singa-site/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #544: SINGA-493 Update news on SINGA tutorial
nudles merged pull request #544: SINGA-493 Update news on SINGA tutorial URL: https://github.com/apache/incubator-singa/pull/544 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #544: SINGA-493 Update news on SINGA tutorial
chrishkchris edited a comment on issue #544: SINGA-493 Update news on SINGA tutorial URL: https://github.com/apache/incubator-singa/pull/544#issuecomment-538370010 I forked the incubator-singa-site and put the preview of website at https://github.com/chrishkchris/incubator-singa-site/tree/SINGA-493 When I download it and check it in microsoft EDGE web browser it is okay This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on issue #544: SINGA-493 Update news on SINGA tutorial
chrishkchris commented on issue #544: SINGA-493 Update news on SINGA tutorial URL: https://github.com/apache/incubator-singa/pull/544#issuecomment-538370010 I forked the incubator-singa-site and put the preview of website at https://github.com/chrishkchris/incubator-singa-site/tree/SINGA-493 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #544: SINGA-493 Update news on singa tutorial
chrishkchris opened a new pull request #544: SINGA-493 Update news on singa tutorial URL: https://github.com/apache/incubator-singa/pull/544 The webpage look like this when I use Sphinx to build:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #541: added 4d test on batchnorm
joddiy commented on issue #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541#issuecomment-538211845 the return of _CpuBatchNormForwardInference_ should be y, var, mean, not just y. could you fix it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #540: added softmax with axis
joddiy commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-538211628 still has some problems, the output of multiple dimension inputs is not correct. please check: ``` x_0 = np.array([[0, 1, 2, 3], [1, 10001, 10002, 10003]]).astype(np.float32) # axis is 1 # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428], # [0.0320586, 0.08714432, 0.23688284, 0.64391428]] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #543: SINGA-491 Use const reference for CopyData and ResetLike from Tensor Input
nudles merged pull request #543: SINGA-491 Use const reference for CopyData and ResetLike from Tensor Input URL: https://github.com/apache/incubator-singa/pull/543 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #543: SINGA-491 Use const reference when we use CopyData and ResetLike from Tensor Input
chrishkchris opened a new pull request #543: SINGA-491 Use const reference when
we use CopyData and ResetLike from Tensor Input
URL: https://github.com/apache/incubator-singa/pull/543
This is to fix the remaining C language alert from LGTM:
There are some functions using ResetLike and CopyData to copy the data from
the tensor input. This PR changes the input type from "Tensor" to "const
Tensor&"
However, it seems that the src/model/layer is used by the old
optimizer/model, it does not have effect on the autograd.py based model.
If I am correct, this avoid the system copying the input tensor with the
tensor copy constructor, instead it uses the address of the input pointer as
reference.
For a verification, the batchnorm in src/model/layer is used by the old
optimizer (not the autograd one), and hence we use the old resnet example to
verify the training loss reduction progress.
```
ubuntu@ip-172-31-42-250:~/incubator-singa/examples/cifar10$ python3 train.py
resnet cifar-10-batches-py
Loading data ..
Loading data file cifar-10-batches-py/data_batch_1
Loading data file cifar-10-batches-py/data_batch_2
Loading data file cifar-10-batches-py/data_batch_3
Loading data file cifar-10-batches-py/data_batch_4
Loading data file cifar-10-batches-py/data_batch_5
Loading data file cifar-10-batches-py/test_batch
('conv1', (16, 32, 32))
('bn1', (16, 32, 32))
('relu1', (16, 32, 32))
('2a-split', [(16, 32, 32), (16, 32, 32)])
('2a-br1-conv1', (16, 32, 32))
('2a-br1-bn1', (16, 32, 32))
('2a-br1-relu', (16, 32, 32))
('2a-br1-conv2', (16, 32, 32))
('2a-br1-bn2', (16, 32, 32))
('2a-merge', [(16, 32, 32), (16, 32, 32)])
('2b-split', [(16, 32, 32), (16, 32, 32)])
('2b-br1-conv1', (16, 32, 32))
('2b-br1-bn1', (16, 32, 32))
('2b-br1-relu', (16, 32, 32))
('2b-br1-conv2', (16, 32, 32))
('2b-br1-bn2', (16, 32, 32))
('2b-merge', [(16, 32, 32), (16, 32, 32)])
('2c-split', [(16, 32, 32), (16, 32, 32)])
('2c-br1-conv1', (16, 32, 32))
('2c-br1-bn1', (16, 32, 32))
('2c-br1-relu', (16, 32, 32))
('2c-br1-conv2', (16, 32, 32))
('2c-br1-bn2', (16, 32, 32))
('2c-merge', [(16, 32, 32), (16, 32, 32)])
('3a-split', [(16, 32, 32), (16, 32, 32)])
('3a-br2-conv', (32, 16, 16))
('3a-br2-bn', (32, 16, 16))
('3a-br1-conv1', (32, 16, 16))
('3a-br1-bn1', (32, 16, 16))
('3a-br1-relu', (32, 16, 16))
('3a-br1-conv2', (32, 16, 16))
('3a-br1-bn2', (32, 16, 16))
('3a-merge', [(32, 16, 16), (32, 16, 16)])
('3b-split', [(32, 16, 16), (32, 16, 16)])
('3b-br1-conv1', (32, 16, 16))
('3b-br1-bn1', (32, 16, 16))
('3b-br1-relu', (32, 16, 16))
('3b-br1-conv2', (32, 16, 16))
('3b-br1-bn2', (32, 16, 16))
('3b-merge', [(32, 16, 16), (32, 16, 16)])
('3c-split', [(32, 16, 16), (32, 16, 16)])
('3c-br1-conv1', (32, 16, 16))
('3c-br1-bn1', (32, 16, 16))
('3c-br1-relu', (32, 16, 16))
('3c-br1-conv2', (32, 16, 16))
('3c-br1-bn2', (32, 16, 16))
('3c-merge', [(32, 16, 16), (32, 16, 16)])
('4a-split', [(32, 16, 16), (32, 16, 16)])
('4a-br2-conv', (64, 8, 8))
('4a-br2-bn', (64, 8, 8))
('4a-br1-conv1', (64, 8, 8))
('4a-br1-bn1', (64, 8, 8))
('4a-br1-relu', (64, 8, 8))
('4a-br1-conv2', (64, 8, 8))
('4a-br1-bn2', (64, 8, 8))
('4a-merge', [(64, 8, 8), (64, 8, 8)])
('4b-split', [(64, 8, 8), (64, 8, 8)])
('4b-br1-conv1', (64, 8, 8))
('4b-br1-bn1', (64, 8, 8))
('4b-br1-relu', (64, 8, 8))
('4b-br1-conv2', (64, 8, 8))
('4b-br1-bn2', (64, 8, 8))
('4b-merge', [(64, 8, 8), (64, 8, 8)])
('4c-split', [(64, 8, 8), (64, 8, 8)])
('4c-br1-conv1', (64, 8, 8))
('4c-br1-bn1', (64, 8, 8))
('4c-br1-relu', (64, 8, 8))
('4c-br1-conv2', (64, 8, 8))
('4c-br1-bn2', (64, 8, 8))
('4c-merge', [(64, 8, 8), (64, 8, 8)])
('pool4', (64, 1, 1))
('flat', (64,))
('ip5', (10,))
Start intialization
Start intialization
Using GPU
Epoch=0:
100%|██|
500/500 [00:17<00:00, 28.18it/s, accuracy=0.59, loss=1.13]
Training loss = 1.418575, training accuracy = 0.481940, lr = 0.10
Test loss = 1.145096, test accuracy = 0.586800
Epoch=1:
100%|█|
500/500 [00:17<00:00, 29.34it/s, accuracy=0.76, loss=0.784]
Training loss = 0.996122, training accuracy = 0.645940, lr = 0.10
Test loss = 0.947394, test accuracy = 0.665900
Epoch=2:
100%|█|
500/500 [00:17<00:00, 28.81it/s, accuracy=0.81, loss=0.696]
Training loss = 0.812576, training accuracy = 0.713660, lr = 0.10
Test loss = 0.830808, test accuracy = 0.713700
Epoch=3:
100%|█|
500/500 [00:17<00:00, 27.96it/s, accuracy=0.81, loss=0.617]
[GitHub] [incubator-singa] lzjpaul opened a new pull request #542: SINGA-492 add Food(lg) news in the documentation
lzjpaul opened a new pull request #542: SINGA-492 add Food(lg) news in the documentation URL: https://github.com/apache/incubator-singa/pull/542 -add Food(lg) news in the SINGA homepage This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin opened a new pull request #541: added 4d test on batchnorm
dcslin opened a new pull request #541: added 4d test on batchnorm URL: https://github.com/apache/incubator-singa/pull/541 added batchnorm test adapted from `/test/python/test_onnx_backend.py` and test passed. Hi @joddiy , If current Batchnorm API could fulfil ONNX batchnorm requirement, shall we keep the current API unchanged? Kindly advise. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin opened a new pull request #540: added softmax with axis
dcslin opened a new pull request #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540 splitting https://github.com/apache/incubator-singa/pull/538 into two parts, this PR is for softmax with axis, which is compatible to onnx api This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #539: SINGA-491 Python Code Cleaning
nudles merged pull request #539: SINGA-491 Python Code Cleaning URL: https://github.com/apache/incubator-singa/pull/539 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #539: SINGA-491 Python Code Cleaning
chrishkchris opened a new pull request #539: SINGA-491 Python Code Cleaning
URL: https://github.com/apache/incubator-singa/pull/539
Only Python code is cleaned at this time that is likely to remove around
hundred alerts in python code.
Since there are lots of changes in the folders examples/imagenet/inception
and examples/imagenet/googlenet, I have run those code again including four
parts:
PART (I) examples/imagenet/googlenet/serve.py
PART (II) examples/imagenet/inception/inception_v3.py
PART (III) examples/imagenet/inception/inception_v4.py
PART (IV) Python Unit Test (test_operation.py)
Two test photos from the imagenet dataset are used for testing in PART (I),
(II), (III)
image1.JPEG
image2.JPEG
The results are okay as expected.
PART (I) examples/imagenet/googlenet/serve.py
```
ubuntu@ip-172-31-39-12:~/incubator-singa/examples/imagenet/googlenet$
python3 serve.py &
runing with gpu
* Serving Flask app "rafiki.agent" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production
deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://0.0.0.0:/ (Press CTRL+C to quit)
Start intialization
('conv1/7x7_s2', (64, 112, 112))
('conv1/relu_7x7', (64, 112, 112))
('pool1/3x3_s2/pad', (64, 113, 113))
('pool1/3x3_s2', (64, 56, 56))
('pool1/norm1', (64, 56, 56))
('conv2/3x3_reduce', (64, 56, 56))
('conv2/relue_3x3_reduce', (64, 56, 56))
('conv2/3x3', (192, 56, 56))
('conv2/relue_3x3', (192, 56, 56))
('conv2/norm2', (192, 56, 56))
('pool2/3x3_s2/pad', (192, 57, 57))
('pool2/3x3_s2', (192, 28, 28))
('inception_3a/split', [(192, 28, 28), (192, 28, 28), (192, 28, 28), (192,
28, 28)])
('inception_3a/1x1', (64, 28, 28))
('inception_3a/relue_1x1', (64, 28, 28))
('inception_3a/3x3_reduce', (96, 28, 28))
('inception_3a/relue_3x3_reduce', (96, 28, 28))
('inception_3a/3x3', (128, 28, 28))
('inception_3a/relue_3x3', (128, 28, 28))
('inception_3a/5x5_reduce', (16, 28, 28))
('inception_3a/relue_5x5_reduce', (16, 28, 28))
('inception_3a/5x5', (32, 28, 28))
('inception_3a/relue_5x5', (32, 28, 28))
('inception_3a/pool', (192, 28, 28))
('inception_3a/pool_proj', (32, 28, 28))
('inception_3a/relue_pool_proj', (32, 28, 28))
('inception_3a/output', (256, 28, 28))
('inception_3b/split', [(256, 28, 28), (256, 28, 28), (256, 28, 28), (256,
28, 28)])
('inception_3b/1x1', (128, 28, 28))
('inception_3b/relue_1x1', (128, 28, 28))
('inception_3b/3x3_reduce', (128, 28, 28))
('inception_3b/relue_3x3_reduce', (128, 28, 28))
('inception_3b/3x3', (192, 28, 28))
('inception_3b/relue_3x3', (192, 28, 28))
('inception_3b/5x5_reduce', (32, 28, 28))
('inception_3b/relue_5x5_reduce', (32, 28, 28))
('inception_3b/5x5', (96, 28, 28))
('inception_3b/relue_5x5', (96, 28, 28))
('inception_3b/pool', (256, 28, 28))
('inception_3b/pool_proj', (64, 28, 28))
('inception_3b/relue_pool_proj', (64, 28, 28))
('inception_3b/output', (480, 28, 28))
('pool3/3x3_s2/pad', (480, 29, 29))
('pool3/3x3_s2', (480, 14, 14))
('inception_4a/split', [(480, 14, 14), (480, 14, 14), (480, 14, 14), (480,
14, 14)])
('inception_4a/1x1', (192, 14, 14))
('inception_4a/relue_1x1', (192, 14, 14))
('inception_4a/3x3_reduce', (96, 14, 14))
('inception_4a/relue_3x3_reduce', (96, 14, 14))
('inception_4a/3x3', (208, 14, 14))
('inception_4a/relue_3x3', (208, 14, 14))
('inception_4a/5x5_reduce', (16, 14, 14))
('inception_4a/relue_5x5_reduce', (16, 14, 14))
('inception_4a/5x5', (48, 14, 14))
('inception_4a/relue_5x5', (48, 14, 14))
('inception_4a/pool', (480, 14, 14))
('inception_4a/pool_proj', (64, 14, 14))
('inception_4a/relue_pool_proj', (64, 14, 14))
('inception_4a/output', (512, 14, 14))
('inception_4b/split', [(512, 14, 14), (512, 14, 14), (512, 14, 14), (512,
14, 14)])
('inception_4b/1x1', (160, 14, 14))
('inception_4b/relue_1x1', (160, 14, 14))
('inception_4b/3x3_reduce', (112, 14, 14))
('inception_4b/relue_3x3_reduce', (112, 14, 14))
('inception_4b/3x3', (224, 14, 14))
('inception_4b/relue_3x3', (224, 14, 14))
('inception_4b/5x5_reduce', (24, 14, 14))
('inception_4b/relue_5x5_reduce', (24, 14, 14))
('inception_4b/5x5', (64, 14, 14))
('inception_4b/relue_5x5', (64, 14, 14))
('inception_4b/pool', (512, 14, 14))
('inception_4b/pool_proj', (64, 14, 14))
('inception_4b/relue_pool_proj', (64, 14, 14))
('inception_4b/output', (512, 14, 14))
('inception_4c/split', [(512, 14, 14), (512, 14, 14), (512, 14, 14), (512,
14, 14)])
('inception_4c/1x1',
[GitHub] [incubator-singa] nudles merged pull request #537: SINGA-488 Change the path of source code for CI
nudles merged pull request #537: SINGA-488 Change the path of source code for CI URL: https://github.com/apache/incubator-singa/pull/537 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin opened a new pull request #538: add Cudnn mode as optional param to batch norm and softmax
dcslin opened a new pull request #538: add Cudnn mode as optional param to batch norm and softmax URL: https://github.com/apache/incubator-singa/pull/538 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #537: SINGA-488 Change the path of source code for CI
chrishkchris opened a new pull request #537: SINGA-488 Change the path of source code for CI URL: https://github.com/apache/incubator-singa/pull/537 This will fix the issue in "SINGA-488 Travis CI always build from Apache master branch" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
nudles merged pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal URL: https://github.com/apache/incubator-singa/pull/528 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result
nudles merged pull request #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result URL: https://github.com/apache/incubator-singa/pull/536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #514: SINGA-482 tc comprehension integration
dcslin commented on issue #514: SINGA-482 tc comprehension integration URL: https://github.com/apache/incubator-singa/pull/514#issuecomment-534047644 Integration is done except support CPU. CPU support development is done, however the behavior was not expected even in clean TC env. CPU backed TC return zero tensor as shown in https://gist.githubusercontent.com/dcslin/f3ed411012c144163f3bedbe621257f6/raw/9f6af3f5e62b5bb52a58c087967c813b3acd8872/tensor-comprehension-cpu-errors This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on issue #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result
chrishkchris commented on issue #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result URL: https://github.com/apache/incubator-singa/pull/536#issuecomment-533585944 Thanks so much for your comments. It is a great idea concerning the CI. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] moazreyad commented on issue #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result
moazreyad commented on issue #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result URL: https://github.com/apache/incubator-singa/pull/536#issuecomment-533497942 Thank you. In future we should not accept only the compile output alone, we should also run the unit tests and make sure they pass with no errors. This must be done on all supported platforms, not only on the developer machine. This is why the Travis continuous integration needs to be improved to automate this task which is very hard to be done manually. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result
chrishkchris opened a new pull request #536: SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result URL: https://github.com/apache/incubator-singa/pull/536 Since LGTM has been applied for our code analysis (see SINGA-484), I have cleaned up many of the obvious issues alerted by LGTM. The compile result is okay as follows: ``` ubuntu@ip-172-31-39-137:~/incubator-singa/build$ rm -rf * ubuntu@ip-172-31-39-137:~/incubator-singa/build$ cmake -D CMAKE_PREFIX_PATH="/usr/local/cuda/lib64;/usr/local/ cuda/" -DENABLE_TEST=OFF -DUSE_CUDA=ON -DUSE_PYTHON3=ON -DUSE_MKLDNN=ON -DUSE_MODULES=OFF -DUSE_DIST=ON .. -- The C compiler identification is GNU 5.4.0 -- The CXX compiler identification is GNU 5.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found suitable version "3.0.0", minimum required i s "3.0") -- Found CBLAS: /usr/local/include -- Found GLOG: /usr/include -- Found cuda_v10.0 -- Found CUDNN: /usr/local/cuda/include -- Found Cudnn_7401 at /usr/local/cuda/include /usr/local/cuda/lib64/libcudnn.so -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3") -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so (found suitable version "3.5.2", minimum requi red is "3") -- Found SWIG: /usr/local/bin/swig (found suitable version "3.0.12", minimum required is "3.0.10") -- Found MKLDNN at /usr/local/include -- Found MPI at /home/ubuntu/mpich-3.3/build/include -- Found MPI lib at /home/ubuntu/mpich-3.3/build/lib/libmpi.so -- Found all lib at /usr/local/lib/libprotobuf.so;/usr/local/lib/libopenblas.so;/usr/lib/x86_64-linux-gnu/libg log.so;/usr/local/cuda/lib64/libcudnn.so;/usr/local/cuda/lib64/libcudart.so;/usr/local/cuda/lib64/libcurand.so ;/usr/local/cuda/lib64/libcublas.so;/home/ubuntu/incubator-singa/build/lib/libcnmem.a;/usr/local/lib/libmkldnn .so;/home/ubuntu/mpich-3.3/build/lib/libmpi.so;/home/ubuntu/mpich-3.3/build/lib/libmpicxx.so -- Found NCCL at /usr/local/cuda/include -- Found NCCL lib at /usr/local/cuda/lib/libnccl.so -- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/incubator-singa/build ubuntu@ip-172-31-39-137:~/incubator-singa/build$ make -j4 Scanning dependencies of target cnmem Scanning dependencies of target copy_protobuf [ 1%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/model.proto [ 2%] Creating directories for 'cnmem' [ 3%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/caffe.proto [ 4%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/core.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: core.prot o. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 s yntax.) [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: model.pro to. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) [ 5%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/io.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: io.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syn tax.) [ 6%] Performing download step (git clone) for 'cnmem' Cloning into 'cnmem'... [ 7%] Copying Protobuf headers [ 7%] Built target copy_protobuf [ 8%] Building NVCC (Device) object
[GitHub] [incubator-singa] nudles merged pull request #529: SINGA-484 Code analysis with LGTM
nudles merged pull request #529: SINGA-484 Code analysis with LGTM URL: https://github.com/apache/incubator-singa/pull/529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #533: fixed operation add assertion for convolution
nudles merged pull request #533: fixed operation add assertion for convolution URL: https://github.com/apache/incubator-singa/pull/533 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
nudles merged pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD) URL: https://github.com/apache/incubator-singa/pull/535 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #533: fixed operation add assertion for convolution
dcslin commented on issue #533: fixed operation add assertion for convolution URL: https://github.com/apache/incubator-singa/pull/533#issuecomment-533383280 Hi @nudles , this is ready for merge. thanks to @chrishkchris This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on a change in pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris commented on a change in pull request #535: SINGA-490 Optimize
performance of stochastic gradient descent (SGD)
URL: https://github.com/apache/incubator-singa/pull/535#discussion_r326004398
##
File path: src/core/tensor/tensor_math_cuda.h
##
@@ -324,12 +324,8 @@ void EltwiseMult(const Tensor& in,
const float x, Tensor* out, Context* ctx) {
const float* inPtr = static_cast(in.block()->data());
float* outPtr = static_cast(out->block()->mutable_data());
-
- float alpha = x, beta = 0.0;
- check_cudnn(cudnnAddTensor(ctx->cudnn_handle,
- (void*)(), generate_tensor_nd_desc(in),
inPtr,
- (void*)(), generate_tensor_nd_desc(*out),
outPtr
-));
+ const size_t num = in.Size();
Review comment:
Yes, must be. It is because cudnnAddTensor add two tensors alphaX+betaY and
write the result to Y. Meanwhile, cuda::mult is for purely elementwise
multiply. The computational cost of cudnnAddTensor is hence much higher than
cuda::mult (when the two inputs are tensor and scalar, respectively). We do not
need to use a tensor add function to perform scalar multiply purpose for the
consideration of computation cost. Our function is just to multiply a tensor
with a scalar.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on a change in pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris commented on a change in pull request #535: SINGA-490 Optimize
performance of stochastic gradient descent (SGD)
URL: https://github.com/apache/incubator-singa/pull/535#discussion_r326006074
##
File path: src/core/tensor/tensor_math_cuda.h
##
@@ -324,12 +324,8 @@ void EltwiseMult(const Tensor& in,
const float x, Tensor* out, Context* ctx) {
const float* inPtr = static_cast(in.block()->data());
float* outPtr = static_cast(out->block()->mutable_data());
-
- float alpha = x, beta = 0.0;
- check_cudnn(cudnnAddTensor(ctx->cudnn_handle,
- (void*)(), generate_tensor_nd_desc(in),
inPtr,
- (void*)(), generate_tensor_nd_desc(*out),
outPtr
-));
+ const size_t num = in.Size();
Review comment:
In other word, we only need alphaX but not alphaX+betaY (while setting
beta=0). The first case has a less computation cost than the latter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on a change in pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris commented on a change in pull request #535: SINGA-490 Optimize
performance of stochastic gradient descent (SGD)
URL: https://github.com/apache/incubator-singa/pull/535#discussion_r326004398
##
File path: src/core/tensor/tensor_math_cuda.h
##
@@ -324,12 +324,8 @@ void EltwiseMult(const Tensor& in,
const float x, Tensor* out, Context* ctx) {
const float* inPtr = static_cast(in.block()->data());
float* outPtr = static_cast(out->block()->mutable_data());
-
- float alpha = x, beta = 0.0;
- check_cudnn(cudnnAddTensor(ctx->cudnn_handle,
- (void*)(), generate_tensor_nd_desc(in),
inPtr,
- (void*)(), generate_tensor_nd_desc(*out),
outPtr
-));
+ const size_t num = in.Size();
Review comment:
Yes, must be. It is because cudnnAddTensor add two tensors alphaX+betaY and
write the result to Y. Meanwhile, cuda::mult is for purely elementwise
multiply. The computational cost of cudnnAddTensor is hence much higher than
cuda::mult. We do not need to use a tensor add function to perform elementwise
multiply purpose for the consideration of computation cost.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] nudles commented on a change in pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
nudles commented on a change in pull request #535: SINGA-490 Optimize
performance of stochastic gradient descent (SGD)
URL: https://github.com/apache/incubator-singa/pull/535#discussion_r326002562
##
File path: src/core/tensor/tensor_math_cuda.h
##
@@ -324,12 +324,8 @@ void EltwiseMult(const Tensor& in,
const float x, Tensor* out, Context* ctx) {
const float* inPtr = static_cast(in.block()->data());
float* outPtr = static_cast(out->block()->mutable_data());
-
- float alpha = x, beta = 0.0;
- check_cudnn(cudnnAddTensor(ctx->cudnn_handle,
- (void*)(), generate_tensor_nd_desc(in),
inPtr,
- (void*)(), generate_tensor_nd_desc(*out),
outPtr
-));
+ const size_t num = in.Size();
Review comment:
is cuda::mult faster than cudnnAddTensor?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris edited a comment on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD) URL: https://github.com/apache/incubator-singa/pull/535#issuecomment-532944885 Since I have done some optimization of the framework, I need to retest again the distributed performance: I tested the distributed training in AWS p2.x8large, after adding the Sync() in the SGD loop of resnet.py and resnet_dist.py. The speed up of using 8 GPUs is now 7.21x, but this is compared without real data feeding. See the following throughput comparison in resnet.py and resnet_dist.py: ``` ubuntu@ip-172-31-28-231:~/incubator-singa/examples/autograd$ python3 resnet.py Start intialization 100%|█| 100/100 [01:23<00:00, 1.19it/s] Throughput = 38.13589358185999 per second Total=0.8391045022010803, forward=0.26401839971542357, softmax=0.0020227289199829103, backward=0.5730633735656739, sgd=0.016838366985321044 ubuntu@ip-172-31-28-231:~/incubator-singa/examples/autograd$ /home/ubuntu/mpich-3.3/build/bin/mpiexec --hostfile host_file python3 resnet_dist.py Start intialization... 100%|██| 100/100 [01:33<00:00, 1.08it/s] Throughput = 274.9947180123401 per second Total=0.9309269714355469, forward=0.2690380573272705, softmax=0.0021610450744628906, backward=0.6597278690338135, sgd=0.10374969005584717 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris commented on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD) URL: https://github.com/apache/incubator-singa/pull/535#issuecomment-532944885 Finally, I test the distributed training in AWS p2.x8large, after adding the Sync() in the SGD loop of resnet.py and resnet_dist.py. The speed up of using 8 GPUs is now 7.21x, but this is compared without real data feeding. See the following throughput comparison in resnet.py and resnet_dist.py: ``` ubuntu@ip-172-31-28-231:~/incubator-singa/examples/autograd$ python3 resnet.py Start intialization 100%|█| 100/100 [01:23<00:00, 1.19it/s] Throughput = 38.13589358185999 per second Total=0.8391045022010803, forward=0.26401839971542357, softmax=0.0020227289199829103, backward=0.5730633735656739, sgd=0.016838366985321044 ubuntu@ip-172-31-28-231:~/incubator-singa/examples/autograd$ /home/ubuntu/mpich-3.3/build/bin/mpiexec --hostfile host_file python3 resnet_dist.py Start intialization... 100%|██| 100/100 [01:33<00:00, 1.08it/s] Throughput = 274.9947180123401 per second Total=0.9309269714355469, forward=0.2690380573272705, softmax=0.0021610450744628906, backward=0.6597278690338135, sgd=0.10374969005584717 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris edited a comment on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD) URL: https://github.com/apache/incubator-singa/pull/535#issuecomment-532723250 Next, I further improve the time performance by using in-place elementwise multiply in many autograd.py functions such as ReLU. Here is the result: ``` ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 mnist_cnn.py Starting Epoch 0: Training loss = 585.42, training accuracy = 0.793390 Evaluation accuracy = 0.939303, Elapsed Time = 4.206943s Starting Epoch 1: Training loss = 234.893921, training accuracy = 0.922409 Evaluation accuracy = 0.955729, Elapsed Time = 4.101450s Starting Epoch 2: Training loss = 169.515244, training accuracy = 0.943286 Evaluation accuracy = 0.970252, Elapsed Time = 4.104907s Starting Epoch 3: Training loss = 136.331894, training accuracy = 0.954442 Evaluation accuracy = 0.968450, Elapsed Time = 4.115959s Starting Epoch 4: Training loss = 118.268318, training accuracy = 0.960512 Evaluation accuracy = 0.971755, Elapsed Time = 4.117009s Starting Epoch 5: Training loss = 104.006439, training accuracy = 0.965732 Evaluation accuracy = 0.978866, Elapsed Time = 4.117350s Starting Epoch 6: Training loss = 93.860809, training accuracy = 0.969067 Evaluation accuracy = 0.977464, Elapsed Time = 4.106471s Starting Epoch 7: Training loss = 88.009178, training accuracy = 0.970251 Evaluation accuracy = 0.982873, Elapsed Time = 4.116037s Starting Epoch 8: Training loss = 81.978348, training accuracy = 0.972802 Evaluation accuracy = 0.983974, Elapsed Time = 4.121274s Starting Epoch 9: Training loss = 75.998878, training accuracy = 0.974103 Evaluation accuracy = 0.982272, Elapsed Time = 4.122591s ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris commented on issue #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD) URL: https://github.com/apache/incubator-singa/pull/535#issuecomment-532723250 Next, I further improve the time performance by using in-place elementwise multiply in many functions such as ReLU. Here is the result: ``` ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 mnist_cnn.py Starting Epoch 0: Training loss = 585.42, training accuracy = 0.793390 Evaluation accuracy = 0.939303, Elapsed Time = 4.206943s Starting Epoch 1: Training loss = 234.893921, training accuracy = 0.922409 Evaluation accuracy = 0.955729, Elapsed Time = 4.101450s Starting Epoch 2: Training loss = 169.515244, training accuracy = 0.943286 Evaluation accuracy = 0.970252, Elapsed Time = 4.104907s Starting Epoch 3: Training loss = 136.331894, training accuracy = 0.954442 Evaluation accuracy = 0.968450, Elapsed Time = 4.115959s Starting Epoch 4: Training loss = 118.268318, training accuracy = 0.960512 Evaluation accuracy = 0.971755, Elapsed Time = 4.117009s Starting Epoch 5: Training loss = 104.006439, training accuracy = 0.965732 Evaluation accuracy = 0.978866, Elapsed Time = 4.117350s Starting Epoch 6: Training loss = 93.860809, training accuracy = 0.969067 Evaluation accuracy = 0.977464, Elapsed Time = 4.106471s Starting Epoch 7: Training loss = 88.009178, training accuracy = 0.970251 Evaluation accuracy = 0.982873, Elapsed Time = 4.116037s Starting Epoch 8: Training loss = 81.978348, training accuracy = 0.972802 Evaluation accuracy = 0.983974, Elapsed Time = 4.121274s Starting Epoch 9: Training loss = 75.998878, training accuracy = 0.974103 Evaluation accuracy = 0.982272, Elapsed Time = 4.122591s ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD)
chrishkchris opened a new pull request #535: SINGA-490 Optimize performance of stochastic gradient descent (SGD) URL: https://github.com/apache/incubator-singa/pull/535 I have fused the small operations of momentum SGD so as to increase GPU computation efficiency and decrease latency. Moreover, I have added the Sync() function for better time profiling in resnet.py (wait for the previous cuda operations to be finished before start calculating the time). 1. This is the new result after improving the momentum SGD: ``` ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 mnist_cnn.py Starting Epoch 0: Training loss = 583.052124, training accuracy = 0.793690 Evaluation accuracy = 0.943409, Elapsed Time = 4.191409s Starting Epoch 1: Training loss = 229.894424, training accuracy = 0.923609 Evaluation accuracy = 0.961438, Elapsed Time = 4.170332s Starting Epoch 2: Training loss = 168.670303, training accuracy = 0.943937 Evaluation accuracy = 0.964744, Elapsed Time = 4.186504s Starting Epoch 3: Training loss = 133.865494, training accuracy = 0.955259 Evaluation accuracy = 0.978566, Elapsed Time = 4.188593s Starting Epoch 4: Training loss = 116.104378, training accuracy = 0.961730 Evaluation accuracy = 0.971554, Elapsed Time = 4.195830s Starting Epoch 5: Training loss = 101.295425, training accuracy = 0.966299 Evaluation accuracy = 0.974059, Elapsed Time = 4.191312s Starting Epoch 6: Training loss = 94.570869, training accuracy = 0.969684 Evaluation accuracy = 0.977464, Elapsed Time = 4.181115s Starting Epoch 7: Training loss = 85.930618, training accuracy = 0.970968 Evaluation accuracy = 0.984675, Elapsed Time = 4.182598s Starting Epoch 8: Training loss = 83.169617, training accuracy = 0.971768 Evaluation accuracy = 0.985076, Elapsed Time = 4.202356s Starting Epoch 9: Training loss = 77.906853, training accuracy = 0.973969 Evaluation accuracy = 0.982372, Elapsed Time = 4.191382s ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 resnet.py Start intialization 100%|███| 100/100 [01:26<00:00, 1.14it/s] Throughput = 36.89267491263885 per second Total=0.8673808574676514, forward=0.2684857630729675, softmax=0.0027115750312805176, backward=0.5961835193634033, sgd=0.03734057664871216 ``` 2. This is the old result before improving the momentum SGD: ``` ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 mnist_cnn.py Starting Epoch 0: Training loss = 581.382263, training accuracy = 0.794974 Evaluation accuracy = 0.934495, Elapsed Time = 5.541576s Starting Epoch 1: Training loss = 233.281906, training accuracy = 0.920808 Evaluation accuracy = 0.953025, Elapsed Time = 5.492121s Starting Epoch 2: Training loss = 169.505447, training accuracy = 0.943503 Evaluation accuracy = 0.971454, Elapsed Time = 5.493372s Starting Epoch 3: Training loss = 136.643906, training accuracy = 0.954309 Evaluation accuracy = 0.975761, Elapsed Time = 5.513660s Starting Epoch 4: Training loss = 116.743042, training accuracy = 0.960963 Evaluation accuracy = 0.979968, Elapsed Time = 5.526858s Starting Epoch 5: Training loss = 103.864464, training accuracy = 0.965732 Evaluation accuracy = 0.979667, Elapsed Time = 5.513694s Starting Epoch 6: Training loss = 94.542282, training accuracy = 0.968550 Evaluation accuracy = 0.975461, Elapsed Time = 5.520474s Starting Epoch 7: Training loss = 87.548050, training accuracy = 0.971368 Evaluation accuracy = 0.980970, Elapsed Time = 5.535038s Starting Epoch 8: Training loss = 83.162071, training accuracy = 0.971485 Evaluation accuracy = 0.975661, Elapsed Time = 5.536836s Starting Epoch 9: Training loss = 78.447533, training accuracy = 0.974570 Evaluation accuracy = 0.982772, Elapsed Time = 5.547574s ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 resnet.py Start intialization 100%|███| 100/100 [01:49<00:00, 1.11s/it] Throughput = 29.05542749993395 per second Total=1.101343286037445, forward=0.270987823009491, softmax=0.0029543495178222657, backward=0.8274011135101318, sgd=0.3130151700973511 ``` From above two sets of results (1) and (2), we can see that the new momentum SGD is much faster after fusing the small operations. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] pinpom commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
pinpom commented on a change in pull request #528: SINGA 475 - add logical
operator: and, or, xor, not & negative, reciprocal
URL: https://github.com/apache/incubator-singa/pull/528#discussion_r325471648
##
File path: python/singa/autograd.py
##
@@ -2522,3 +2523,144 @@ def backward(self, dy):
def max(a,b):
return Max()(a,b)[0]
+
+class And(Operation):
+def __init__(self):
+super(And, self).__init__()
+
+def forward(self, a, b):
+m = singa.__div__(a,b)
+
+mask0 = singa.GEFloat(m,1)
+mask1 = singa.LEFloat(m,1)
+cur = singa.__mul__(mask0,mask1)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient')
+return None
+
+def _and(a,b):
+return And()(a,b)[0]
+
+
+class Or(Operation):
+def __init__(self):
+super(Or, self).__init__()
+
+def forward(self, a, b):
+#find equal element-wise
+m = singa.__sub__(a,b)
Review comment:
ok got it. just changed the code already.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] joddiy commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
joddiy commented on a change in pull request #528: SINGA 475 - add logical
operator: and, or, xor, not & negative, reciprocal
URL: https://github.com/apache/incubator-singa/pull/528#discussion_r325462186
##
File path: python/singa/autograd.py
##
@@ -2522,3 +2523,144 @@ def backward(self, dy):
def max(a,b):
return Max()(a,b)[0]
+
+class And(Operation):
+def __init__(self):
+super(And, self).__init__()
+
+def forward(self, a, b):
+m = singa.__div__(a,b)
+
+mask0 = singa.GEFloat(m,1)
+mask1 = singa.LEFloat(m,1)
+cur = singa.__mul__(mask0,mask1)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient')
+return None
+
+def _and(a,b):
+return And()(a,b)[0]
+
+
+class Or(Operation):
+def __init__(self):
+super(Or, self).__init__()
+
+def forward(self, a, b):
+#find equal element-wise
+m = singa.__sub__(a,b)
Review comment:
if a = -b, the Signs of them are, 1, and -1, and then, you take a square on
it, the result will be 1 and 1, and you add them together, is 2, take the Sign
again, you get a 1 finally.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] pinpom commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
pinpom commented on a change in pull request #528: SINGA 475 - add logical
operator: and, or, xor, not & negative, reciprocal
URL: https://github.com/apache/incubator-singa/pull/528#discussion_r325461738
##
File path: python/singa/autograd.py
##
@@ -2522,3 +2523,144 @@ def backward(self, dy):
def max(a,b):
return Max()(a,b)[0]
+
+class And(Operation):
+def __init__(self):
+super(And, self).__init__()
+
+def forward(self, a, b):
+m = singa.__div__(a,b)
+
+mask0 = singa.GEFloat(m,1)
+mask1 = singa.LEFloat(m,1)
+cur = singa.__mul__(mask0,mask1)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient')
+return None
+
+def _and(a,b):
+return And()(a,b)[0]
+
+
+class Or(Operation):
+def __init__(self):
+super(Or, self).__init__()
+
+def forward(self, a, b):
+#find equal element-wise
+m = singa.__sub__(a,b)
Review comment:
i think the equation is not correct if a = -b (element-wise)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] joddiy commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
joddiy commented on a change in pull request #528: SINGA 475 - add logical
operator: and, or, xor, not & negative, reciprocal
URL: https://github.com/apache/incubator-singa/pull/528#discussion_r325123236
##
File path: python/singa/autograd.py
##
@@ -2522,3 +2523,144 @@ def backward(self, dy):
def max(a,b):
return Max()(a,b)[0]
+
+class And(Operation):
+def __init__(self):
+super(And, self).__init__()
+
+def forward(self, a, b):
+m = singa.__div__(a,b)
+
+mask0 = singa.GEFloat(m,1)
+mask1 = singa.LEFloat(m,1)
+cur = singa.__mul__(mask0,mask1)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient')
+return None
+
+def _and(a,b):
+return And()(a,b)[0]
+
+
+class Or(Operation):
+def __init__(self):
+super(Or, self).__init__()
+
+def forward(self, a, b):
+#find equal element-wise
+m = singa.__sub__(a,b)
+m0 = singa.GEFloat(m,0)
+m1 = singa.LEFloat(m,0)
+mask0 = singa.__mul__(m0, m1)
+
+#find 0 element-wise
+n = singa.__add__(a,b)
+n0 = singa.GEFloat(n,0)
+n1 = singa.LEFloat(n,0)
+mask1 = singa.__mul__(n0, n1)
+
+#find equal 0 element-wise
+n = singa.__mul__(mask0, mask1)
+cur = singa.LEFloat(n, 0)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient for backward function')
+return None
+
+
+def _or(a,b):
+return Or()(a,b)[0]
+
+
+class Not(Operation):
+def __init__(self):
+super(Not, self).__init__()
+
+def forward(self, x):
+mask0 = singa.GEFloat(x,0)
+mask1 = singa.LEFloat(x,0)
+cur = singa.__mul__(mask0,mask1)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient for backward function')
+return None
+
+def _not(x):
+return Not()(x)[0]
+
+
+class Xor(Operation):
+def __init__(self):
+super(Xor, self).__init__()
+
+def forward(self, a, b):
+#find element with value =0
+m0 = singa.__mul__(a,b)
Review comment:
not clear, plz use this:
```
m = singa.__sub__(singa.PowFloat(singa.Sign(a), 2.0),
singa.PowFloat(singa.Sign(b), 2.0))
cur = singa.PowFloat(singa.Sign(m), 2.0)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] joddiy commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
joddiy commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal URL: https://github.com/apache/incubator-singa/pull/528#discussion_r325107434 ## File path: python/singa/autograd.py ## @@ -2522,3 +2523,144 @@ def backward(self, dy): def max(a,b): return Max()(a,b)[0] + +class And(Operation): +def __init__(self): +super(And, self).__init__() + +def forward(self, a, b): +m = singa.__div__(a,b) Review comment: This part cannot pass the cpu test case, plz use this. ``` m = singa.__mul__(a, b) cur = singa.PowFloat(singa.Sign(m), 2) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on a change in pull request #528: SINGA 475 - add logical operator: and, or, xor, not & negative, reciprocal
joddiy commented on a change in pull request #528: SINGA 475 - add logical
operator: and, or, xor, not & negative, reciprocal
URL: https://github.com/apache/incubator-singa/pull/528#discussion_r325108637
##
File path: python/singa/autograd.py
##
@@ -2522,3 +2523,144 @@ def backward(self, dy):
def max(a,b):
return Max()(a,b)[0]
+
+class And(Operation):
+def __init__(self):
+super(And, self).__init__()
+
+def forward(self, a, b):
+m = singa.__div__(a,b)
+
+mask0 = singa.GEFloat(m,1)
+mask1 = singa.LEFloat(m,1)
+cur = singa.__mul__(mask0,mask1)
+
+return cur
+
+def backward(self, dy):
+assert 0,('no gradient')
+return None
+
+def _and(a,b):
+return And()(a,b)[0]
+
+
+class Or(Operation):
+def __init__(self):
+super(Or, self).__init__()
+
+def forward(self, a, b):
+#find equal element-wise
+m = singa.__sub__(a,b)
Review comment:
not clear, plz use this:
```
m = singa.__add__(singa.PowFloat(singa.Sign(a), 2.0),
singa.PowFloat(singa.Sign(b), 2.0))
cur = singa.Sign(m)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] dcslin commented on a change in pull request #533: fixed operation add assertion for convolution
dcslin commented on a change in pull request #533: fixed operation add
assertion for convolution
URL: https://github.com/apache/incubator-singa/pull/533#discussion_r324502113
##
File path: src/core/tensor/tensor_math_cpp.h
##
@@ -868,6 +868,25 @@ void RowMax(const Tensor& in, Tensor
*out, Context *ctx) {
}
}
+template <>
+void SoftMax(const Tensor& in, Tensor* out, Context *ctx) {
Review comment:
Hi @nudles , it is not implemented in cpp yet. So we need to move this from
`tensor.cc` into `tensor_math_cpp.h`. If not, when calling softmax with cpp, it
will throw `not impl` fatal error.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] dcslin commented on a change in pull request #533: fixed operation add assertion for convolution
dcslin commented on a change in pull request #533: fixed operation add
assertion for convolution
URL: https://github.com/apache/incubator-singa/pull/533#discussion_r324502113
##
File path: src/core/tensor/tensor_math_cpp.h
##
@@ -868,6 +868,25 @@ void RowMax(const Tensor& in, Tensor
*out, Context *ctx) {
}
}
+template <>
+void SoftMax(const Tensor& in, Tensor* out, Context *ctx) {
Review comment:
Hi @nudles , it is not implemented in cpp yet. So we need to move this from
`tensor.cc` into `tensor_math_cpp.h`. If not, when calling softmax with cpp, it
will through `not impl` fatal error.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #462: SINGA-333 Add ONNX backend test
joddiy commented on issue #462: SINGA-333 Add ONNX backend test URL: https://github.com/apache/incubator-singa/pull/462#issuecomment-531490830 > But if we can't run onnx backend test cases, then we simply can't claim that we support onnx backend. > > We don't have to pass all the test cases, but at least some of them should pass. Currently [all of the test cases fail](https://issues.apache.org/jira/projects/SINGA/issues/SINGA-458), which can be understood as if the current support of onnx backend is 0%. Hi, moazreyad, I know what you mean. Please check the link [here](https://github.com/apache/incubator-singa/pull/534). I mean you cannot just easily call all the onnx test cases, it cannot run. I have adapted the onnx backend test cases from onnx there, and discard the cases that we cannot support. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] moazreyad commented on issue #462: SINGA-333 Add ONNX backend test
moazreyad commented on issue #462: SINGA-333 Add ONNX backend test URL: https://github.com/apache/incubator-singa/pull/462#issuecomment-531487825 But if we can't run onnx backend test cases, then we simply can't claim that we support onnx backend. We don't have to pass all the test cases, but at least some of them should pass. Currently [all of the test cases fail](https://issues.apache.org/jira/projects/SINGA/issues/SINGA-458), which can be understood as if the current support of onnx backend is 0%. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] moazreyad commented on a change in pull request #529: SINGA-484 Code analysis with LGTM
moazreyad commented on a change in pull request #529: SINGA-484 Code analysis
with LGTM
URL: https://github.com/apache/incubator-singa/pull/529#discussion_r324426747
##
File path: include/singa/io/transformer.h
##
@@ -56,12 +56,12 @@ class ImageTransformer: public Transformer {
Tensor Apply(int flag, Tensor& input) override;
- const bool featurewise_center() const { return featurewise_center_; }
- const bool featurewise_std_norm() const { return featurewise_std_norm_; }
- const bool horizontal_mirror() const { return horizontal_mirror_; }
- const int resize_height() const { return resize_height_; }
- const int resize_width() const { return resize_width_; }
- const float rescale() const { return rescale_; }
+ bool featurewise_center() const { return featurewise_center_; }
Review comment:
Yes, LGTM suggested to the remove the const because it is useless. More
information can be found [here](https://lgtm.com/rules/2156200647/).
There are more alerts that I could try to fix, but because the [testing
pipeline is not working](https://issues.apache.org/jira/browse/SINGA-488), I
can't be sure that my fixes will not cause other problems.
All the alerts can be found in this
[link](https://lgtm.com/projects/g/apache/incubator-singa/alerts/?mode=list),
which is also available in the Jira ticket
[484](https://issues.apache.org/jira/browse/SINGA-484)
In this pull request, I propose to add the code analysis badges to SINGA
readme and I fixed few alerts. But there are many more to be fixed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #531: SINGA-485 Add code coverage with codecov
nudles merged pull request #531: SINGA-485 Add code coverage with codecov URL: https://github.com/apache/incubator-singa/pull/531 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles commented on a change in pull request #529: SINGA-484 Code analysis with LGTM
nudles commented on a change in pull request #529: SINGA-484 Code analysis with
LGTM
URL: https://github.com/apache/incubator-singa/pull/529#discussion_r324411358
##
File path: include/singa/io/transformer.h
##
@@ -56,12 +56,12 @@ class ImageTransformer: public Transformer {
Tensor Apply(int flag, Tensor& input) override;
- const bool featurewise_center() const { return featurewise_center_; }
- const bool featurewise_std_norm() const { return featurewise_std_norm_; }
- const bool horizontal_mirror() const { return horizontal_mirror_; }
- const int resize_height() const { return resize_height_; }
- const int resize_width() const { return resize_width_; }
- const float rescale() const { return rescale_; }
+ bool featurewise_center() const { return featurewise_center_; }
Review comment:
Does the code analysis suggest to remove the const?
Where can we find all the suggestions?
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #462: SINGA-333 Add ONNX backend test
joddiy commented on issue #462: SINGA-333 Add ONNX backend test URL: https://github.com/apache/incubator-singa/pull/462#issuecomment-531445928 please refer to [here](https://github.com/apache/incubator-singa/pull/534) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #534: Test onnx backend
joddiy commented on issue #534: Test onnx backend URL: https://github.com/apache/incubator-singa/pull/534#issuecomment-531445904 Please don't merge this PR until I finish all the test cases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy opened a new pull request #534: Test onnx backend
joddiy opened a new pull request #534: Test onnx backend URL: https://github.com/apache/incubator-singa/pull/534 add onnx backend test cases This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] joddiy commented on issue #462: SINGA-333 Add ONNX backend test
joddiy commented on issue #462: SINGA-333 Add ONNX backend test URL: https://github.com/apache/incubator-singa/pull/462#issuecomment-531445841 I think we cannot directly use all the onnx test cases. Since, first, it's hard to integrate the sonnx into it. Second, there are lots of test cases we cannot use because we don't have some functionalities, such as dilation... So far, I've written a new test file to adapt the test cases from onnx, so that we can discard some cases that we cannot implement. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #532: SINGA-487 Add the support of Python Multiprocess Module
nudles merged pull request #532: SINGA-487 Add the support of Python Multiprocess Module URL: https://github.com/apache/incubator-singa/pull/532 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles commented on issue #462: SINGA-333 Add ONNX backend test
nudles commented on issue #462: SINGA-333 Add ONNX backend test URL: https://github.com/apache/incubator-singa/pull/462#issuecomment-531445209 @joddiy can we merge this one? or we need to wait until the operators are all implemented? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles commented on a change in pull request #533: fixed operation add assertion for convolution
nudles commented on a change in pull request #533: fixed operation add
assertion for convolution
URL: https://github.com/apache/incubator-singa/pull/533#discussion_r324410807
##
File path: src/core/tensor/tensor_math_cpp.h
##
@@ -868,6 +868,25 @@ void RowMax(const Tensor& in, Tensor
*out, Context *ctx) {
}
}
+template <>
+void SoftMax(const Tensor& in, Tensor* out, Context *ctx) {
Review comment:
do we need to move SoftMax into tensor_math_cpp.h?
If not, please delete the changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #533: fixed operation add assertion for convolution
dcslin commented on issue #533: fixed operation add assertion for convolution URL: https://github.com/apache/incubator-singa/pull/533#issuecomment-530344769 after the fix, testing on `python/resnet.py` shows similar result: Before: ``` 100%|###| 100/100 [00:27<00:00, 3.68it/s] Throughput = 58.903093484671295 per second Total=0.27163259267807005, forward=0.028347208499908447, softmax=0.10690386295318603, backward=0.13638152122497554, sgd=0.08405628204345703 ``` After: ``` 100%|###| 100/100 [00:27<00:00, 3.69it/s] Throughput = 59.02303205471914 per second Total=0.2710806179046631, forward=0.02830531120300293, softmax=0.10648007154464721, backward=0.13629523515701297, sgd=0.08383089303970337 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #533: fixed operation add assertion for convolution
dcslin commented on issue #533: fixed operation add assertion for convolution URL: https://github.com/apache/incubator-singa/pull/533#issuecomment-530340346 2nd commit fixed that "cudnn softmax is not used" mentioned in https://github.com/apache/incubator-singa/pull/468#issuecomment-521247206 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin opened a new pull request #533: fixed operation add assertion for convolution
dcslin opened a new pull request #533: fixed operation add assertion for convolution URL: https://github.com/apache/incubator-singa/pull/533 this is required to run the `autogra/resnet.py` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris opened a new pull request #532: SINGA-487 Add the support of Python Multiprocess Module
chrishkchris opened a new pull request #532: SINGA-487 Add the support of Python Multiprocess Module URL: https://github.com/apache/incubator-singa/pull/532 I have added the support for the Python Multiprocess Module for single-node multi-gpu scanerio. For the old MPI-based NCCL, I have also simplified and cleaned the code, where I removed the nDev variable (i.e. number of GPU controlled by each process) which is always 1 in our case. So, in the following autograd example, (i) mnist_multiprocess.py is the example using python multiprocessing module (ii) mnist_dist.py is the example using MPI for multiprocessing The results for both examples are as follows: ubuntu@ip-172-31-38-62:~/incubator-singa/examples/autograd$ python3 mnist_multiprocess.py Starting Epoch 0: Training loss = 801.480042, training accuracy = 0.709101 Evaluation accuracy = 0.920436, Elapsed Time = 1.248269s Starting Epoch 1: Training loss = 249.743988, training accuracy = 0.916817 Evaluation accuracy = 0.956620, Elapsed Time = 1.226179s Starting Epoch 2: Training loss = 175.276443, training accuracy = 0.942258 Evaluation accuracy = 0.970498, Elapsed Time = 1.181269s Starting Epoch 3: Training loss = 144.092194, training accuracy = 0.951289 Evaluation accuracy = 0.968236, Elapsed Time = 1.168137s Starting Epoch 4: Training loss = 116.727524, training accuracy = 0.961221 Evaluation accuracy = 0.977282, Elapsed Time = 1.169854s Starting Epoch 5: Training loss = 105.698898, training accuracy = 0.964577 Evaluation accuracy = 0.979132, Elapsed Time = 1.174284s Starting Epoch 6: Training loss = 94.009590, training accuracy = 0.968616 Evaluation accuracy = 0.976460, Elapsed Time = 1.172847s Starting Epoch 7: Training loss = 87.892418, training accuracy = 0.970419 Evaluation accuracy = 0.979852, Elapsed Time = 1.172124s Starting Epoch 8: Training loss = 82.783676, training accuracy = 0.972306 Evaluation accuracy = 0.983141, Elapsed Time = 1.163122s Starting Epoch 9: Training loss = 76.629707, training accuracy = 0.974576 Evaluation accuracy = 0.978927, Elapsed Time = 1.160587s ubuntu@ip-172-31-38-62:~/incubator-singa/examples/autograd$ /home/ubuntu/mpich-3.3/build/bin/mpiexec --hostfile host_file python3 mnist_dist.py Starting Epoch 0: Training loss = 792.865723, training accuracy = 0.713041 Evaluation accuracy = 0.929174, Elapsed Time = 1.262597s Starting Epoch 1: Training loss = 250.669327, training accuracy = 0.914931 Evaluation accuracy = 0.960218, Elapsed Time = 1.198090s Starting Epoch 2: Training loss = 174.226135, training accuracy = 0.941273 Evaluation accuracy = 0.966283, Elapsed Time = 1.189961s Starting Epoch 3: Training loss = 142.276245, training accuracy = 0.952541 Evaluation accuracy = 0.970806, Elapsed Time = 1.189858s Starting Epoch 4: Training loss = 121.220009, training accuracy = 0.959769 Evaluation accuracy = 0.972759, Elapsed Time = 1.190380s Starting Epoch 5: Training loss = 111.639114, training accuracy = 0.962423 Evaluation accuracy = 0.975946, Elapsed Time = 1.186215s Starting Epoch 6: Training loss = 96.729469, training accuracy = 0.967448 Evaluation accuracy = 0.982216, Elapsed Time = 1.177556s Starting Epoch 7: Training loss = 89.441696, training accuracy = 0.970169 Evaluation accuracy = 0.978824, Elapsed Time = 1.183380s Starting Epoch 8: Training loss = 79.853104, training accuracy = 0.973057 Evaluation accuracy = 0.982113, Elapsed Time = 1.181502s Starting Epoch 9: Training loss = 77.974480, training accuracy = 0.974259 Evaluation accuracy = 0.978516, Elapsed Time = 1.183578s This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] nudles merged pull request #468: Distributted module
nudles merged pull request #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] moazreyad merged pull request #426: SINGA-405 Add roles to contributors
moazreyad merged pull request #426: SINGA-405 Add roles to contributors URL: https://github.com/apache/incubator-singa/pull/426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-528186487 Therefore, I suggest we may change the input shape assertion of the ADD function in this way: if the input shapes of the two operands are the same, it also passes the assertion: `assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or (len(self.shape0) == len(self.shape1)) )` If this amendment is okay, I will add this change into this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-528186487 Therefore, I suggest we may change the input shape assertion of the ADD function in this way: if the input shapes of the two operands are the same, it also passes the assertion: ```python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or (len(self.shape0) == len(self.shape1)) ) ` If this amendment is okay, I will add this change into this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-528186487 Therefore, I suggest we may change the input shape assertion of the ADD function in this way: if the input shapes of the two operands are the same, it also passes the assertion: ```python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or (len(self.shape0) == len(self.shape1)) )``` If this amendment is okay, I will add this change into this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-528186487 Therefore, I suggest we may change the input shape assertion of the ADD function in this way: if the input shapes of the two operands are the same, it also passes: ```python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or (len(self.shape0) == len(self.shape1)) ) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-528186487 Therefore, I suggest we may change the input shape assertion of the ADD function in this way: if the input shapes of the two operands are the same, it also passes the assertion: ```python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or (len(self.shape0) == len(self.shape1)) ) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris commented on issue #468: Distributted module
chrishkchris commented on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-528186487 Therefore, I suggest we may change the input shape assertion of the ADD function in this way: if the input shapes of the two operands are The the same, it also passes: ```python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or (len(self.shape0) == len(self.shape1)) ) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or we can change the assert in this way: if the input shapes of the two
operands are the same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0) == len(self.shape1)) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are the same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0) == len(self.shape1)) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0) == len(self.shape1)) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0)==len(self.shape1)) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0)==len(self.shape1)) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert ( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0)==len(self.shape1)) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) or
(len(self.shape0)-len(self.shape1)==0) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) and
(len(self.shape0)-len(self.shape1)!=0) )```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python
assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) and
(len(self.shape0)-len(self.shape1)!=0) )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
```python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) and
(len(self.shape0)-len(self.shape1)!=0) )```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
python `assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) and
(len(self.shape0)-len(self.shape1)!=0) )`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
`python assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) and
(len(self.shape0)-len(self.shape1)!=0) )`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
chrishkchris edited a comment on issue #468: Distributted module
URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-527824025
> I have combined all the commits into two commits. Meanwhile, I found that
the resnet.py is not compatible with the master branch modified "Add" function
with broadcasting. Get the error (assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2),"up till now, the dimensions of tensor a and b should
less than 3"
> AssertionError: up till now, the dimensions of tensor a and b should less
than 3)
> Since in resnet we used "out = autograd.add(out, residual)", the input to
the add function should have a dimension more than 2, the assert function
should return always false and hence assertion error
When I disable the assertion `assert(len(self.shape0) <= 2 and
len(self.shape1) <= 2)`, the resnet.py can run successfully
See the code of Add function
```python
class Add(Operation):
def __init__(self):
super(Add, self).__init__()
def forward(self, a, b):
#up till now, the dimensions of tensor a and b should less than 3
self.shape0=list(a.shape())
self.shape1=list(b.shape())
assert(len(self.shape0) <= 2 and len(self.shape1) <= 2),"up till
now, the dimensions of tensor a and b should less than 3"
return singa.__add__(a, b)
def backward(self, dy):
if(type(dy)==float):
assert self.shape0==self.shape1,('should have same shape')
return dy,dy
db=CTensor(list(dy.shape()), dy.device())
db.CopyData(dy)
for i in range(len(self.shape0)-len(self.shape1)):
db=singa.Sum(db, 0)
return dy, db
```
Can we allow input dimension to ADD more than two? (e.g. change the limit 2
to 4 or disable the assertion). Typically there are four dimensions for conv.
feature maps: batch, depth/channel, width, height
Or if the input shapes of the two operands are two same, it also passes:
`assert( (len(self.shape0) <= 2 and len(self.shape1) <= 2) and
(len(self.shape0)-len(self.shape1)!=0) )`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
