[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519876368 there are conflicts and there are two solutions (same as what happened in log operator): 1. merge this branch with the latest master 2. checkout a new branch from the latest master. Then copy the added code into the autograd.py and test_operations.py; send a new PR. (I guess this maybe because different PRs put the new functions at the same line, so git detected it as different ways of changing code and hence conflicts) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519876368 there are conflicts and there are two solutions (same as what happened in log operator): 1. merge this branch with the latest master 2. checkout a new branch from the latest master. Then copy the added code into the autograd.py and test_operations.py; send a new PR. (I guess this maybe because different PRs put the new functions at the same line, so git detected it as conflicts) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to
singa
URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519866518
This is what tensorflow uses:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/cc/gradients/math_grad.cc#L268
```cpp
Status SignGrad(const Scope& scope, const Operation& op,
const std::vector& grad_inputs,
std::vector* grad_outputs) {
auto shape = Shape(scope, op.input(0));
auto zero = Cast(scope, Const(scope, 0.0), op.input(0).type());
auto dx = Fill(scope, shape, zero);
grad_outputs->push_back(dx);
return scope.status();
}
REGISTER_GRADIENT_OP("Sign", SignGrad);
```
seems to fill the dx with all zero using the dy shape
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to
singa
URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519866518
This is what tensorflow use:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/cc/gradients/math_grad.cc#L268
```cpp
Status SignGrad(const Scope& scope, const Operation& op,
const std::vector& grad_inputs,
std::vector* grad_outputs) {
auto shape = Shape(scope, op.input(0));
auto zero = Cast(scope, Const(scope, 0.0), op.input(0).type());
auto dx = Fill(scope, shape, zero);
grad_outputs->push_back(dx);
return scope.status();
}
REGISTER_GRADIENT_OP("Sign", SignGrad);
```
seems to fill the dx with all zero using the dy shape
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to
singa
URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519866518
This is what tensorflow use:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/cc/gradients/math_grad.cc#L268
```cpp
Status SignGrad(const Scope& scope, const Operation& op,
const std::vector& grad_inputs,
std::vector* grad_outputs) {
auto shape = Shape(scope, op.input(0));
auto zero = Cast(scope, Const(scope, 0.0), op.input(0).type());
auto dx = Fill(scope, shape, zero);
grad_outputs->push_back(dx);
return scope.status();
}
REGISTER_GRADIENT_OP("Sign", SignGrad);
```
seems to fill the dx with all zero with the dy shape
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to
singa
URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519866518
This is what tensorflow use:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/cc/gradients/math_grad.cc#L268
```cpp
Status SignGrad(const Scope& scope, const Operation& op,
const std::vector& grad_inputs,
std::vector* grad_outputs) {
auto shape = Shape(scope, op.input(0));
auto zero = Cast(scope, Const(scope, 0.0), op.input(0).type());
auto dx = Fill(scope, shape, zero);
grad_outputs->push_back(dx);
return scope.status();
}
REGISTER_GRADIENT_OP("Sign", SignGrad);
```
seems to fill the dx with all zero
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-singa] chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519856801 In my opinion: For the function y=sign(x) , y = 1 when x > 0, y = -1 when x < 0. Therefore, the derivative is 0 except x = 0. The function y=sign(x) is discontinuous at x=0 where the derivative is undefined in the mathematical logic. However, an undefined derivative leads to a nan number, so we used a derivative of zero also at x=0. As a result, dx is always 0 while the output shape (size of array) is the same as input shape. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
