[jira] [Updated] (SINGA-494) Singa autograd improvement
[ https://issues.apache.org/jira/browse/SINGA-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangzhaoqi updated SINGA-494: -- Description: Background: some autograd ops cannot satisfy the onnx demand, as following: # *conv, averagepool, maxpool* - only support 2d input, i.e, N*C*W*H - not support SAME_UPPER, SAME_LOWER, count_include_pad and ceil_mod # *reshape* - not support zero_dim, zero_and_negative_dim # *concat* - not support 1d # *matmul* - only support 2d # *min, max* - only support 2 inputs # *add* - not support broadcast # *and, or, xor* - not support broadcast # *div, pow, prelu* - not support broadcast Some improvements are being done. was: Background: some autograd ops cannot satisfy the onnx demand, as following: # conv, averagepool, maxpool - only support 2d input, i.e, N*C*W*H - not support SAME_UPPER, SAME_LOWER, count_include_pad and ceil_mode # reshape - not support zero_dim, zero_and_negative_dim # concat - not support 1d # matmul - only support 2d # min, max - only support 2 inputs # add - not support broadcast # and, or, xor - not support broadcast # div, pow, prelu - not support broadcast Some improvements are being done. > Singa autograd improvement > -- > > Key: SINGA-494 > URL: https://issues.apache.org/jira/browse/SINGA-494 > Project: Singa > Issue Type: New Feature >Reporter: zhangzhaoqi >Priority: Major > > Background: some autograd ops cannot satisfy the onnx demand, as following: > # *conv, averagepool, maxpool* > - only support 2d input, i.e, N*C*W*H > - not support SAME_UPPER, SAME_LOWER, count_include_pad and ceil_mod > # *reshape* > - not support zero_dim, zero_and_negative_dim > # *concat* > - not support 1d > # *matmul* > - only support 2d > # *min, max* > - only support 2 inputs > # *add* > - not support broadcast > # *and, or, xor* > - not support broadcast > # *div, pow, prelu* > - not support broadcast > Some improvements are being done. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-singa] joddiy commented on issue #540: added softmax with axis
joddiy commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-538984488 > > still has some problems, the output of multiple dimension inputs is not correct. > > please check: > > ``` > > x_0 = np.array([[0, 1, 2, 3], [1, 10001, 10002, 10003]]).astype(np.float32) > > # axis is 1 > > # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428], > > # [0.0320586, 0.08714432, 0.23688284, 0.64391428]] > > ``` > > Hi @joddiy , this is updated, could you please help to review? Hi shicong, the axis and output are almost correct. However, the result will overflow if the input values are too big. For example, for input of [0, 1, 2, 3], the result is correct. But for input of 1, 10001, 10002, 10003], the result will be [nan, nan, nan, nan]. Please use this formation: ``` def softmax(x): """Compute softmax values for each sets of scores in x.""" e_x = np.exp(x - np.max(x)) return e_x / e_x.sum() ``` **not directly use x as input, instead subtracting the max value of x from x to avoid the overflow.** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-singa] dcslin commented on issue #540: added softmax with axis
dcslin commented on issue #540: added softmax with axis URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-538955169 > still has some problems, the output of multiple dimension inputs is not correct. > please check: > > ``` > x_0 = np.array([[0, 1, 2, 3], [1, 10001, 10002, 10003]]).astype(np.float32) > # axis is 1 > # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428], > # [0.0320586, 0.08714432, 0.23688284, 0.64391428]] > ``` Hi @joddiy , this is updated, could you please help to review? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
