Repository: systemml Updated Branches: refs/heads/gh-pages 36e9f1f21 -> 781fa43dc
[SYSTEMML-1872] Added average pooling and upsampling layers - Added avg_pool and avg_pool_backward builtin functions. - The above builtin functions are supported on both CPU and GPU. - Also, added compare_backend tests for the above function to compare the results of CP operator with that of CuDNN. - Added avg_pool2d_builtin.dml and upsample2d.dml in the nn library. - Added gradcheck tests for the above layers. - Supported average pooling and upsampling in Keras2DML as well as Caffe2DML. - Tested the results of ResNet with average pooling on real-world images as a sanity check. - Also, tested upsampling layer by comparing the results with that returned by Keras. Closes #734. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/781fa43d Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/781fa43d Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/781fa43d Branch: refs/heads/gh-pages Commit: 781fa43dc8a729ad9a1dac1ac16cef2a3f66364d Parents: 36e9f1f Author: Niketan Pansare <[email protected]> Authored: Tue Feb 27 10:20:08 2018 -0800 Committer: Niketan Pansare <[email protected]> Committed: Tue Feb 27 10:21:57 2018 -0800 ---------------------------------------------------------------------- dml-language-reference.md | 18 +++++++++--------- reference-guide-caffe2dml.md | 26 +++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/systemml/blob/781fa43d/dml-language-reference.md ---------------------------------------------------------------------- diff --git a/dml-language-reference.md b/dml-language-reference.md index 355b507..d0943d6 100644 --- a/dml-language-reference.md +++ b/dml-language-reference.md @@ -1509,15 +1509,15 @@ The images are assumed to be stored NCHW format, where N = batch size, C = #chan Hence, the images are internally represented as a matrix with dimension (N, C * H * W). -| Function name | Input matrices | Dimension of first input matrix | Dimension of second input matrix (if applicable) | Dimension of output matrix | Input Parameters | Notes | -|----------------------------|----------------|-----------------------------------------------------------|-----------------------------------------------------------|------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------| -| conv2d | input, filter | [batch_size X num_channels* height_image* width_image] | [num_filters X num_channels* height_filter* width_filter] | [batch_size X num_channels_out* height_out* width_out] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], filter_shape=[num_filters, num_channels, height_filter, width_filter] | Performs 2D convolution operation | -| conv2d_backward_filter | input, dout | [batch_size X num_channels* height_image* width_image] | [batch_size X num_channels_out* height_out* width_out] | [num_filters X num_channels* height_filter* width_filter] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], filter_shape=[num_filters, num_channels, height_filter, width_filter] | Computes the gradients wrt filter of 2D convolution | -| conv2d_backward_data | filter, dout | [num_filters X num_channels* height_filter* width_filter] | [batch_size X num_channels_out* height_out* width_out] | [batch_size X num_channels* height_image* width_image] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], filter_shape=[num_filters, num_channels, height_filter, width_filter] | Computes the gradients wrt input of 2D convolution | -| max_pool | input | [batch_size X num_channels* height_image* width_image] | | [batch_size X num_channels* height_out* width_out] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], pool_size=[height_pool, width_pool] | Performs max pooling operation | -| max_pool_backward | input, dout | [batch_size X num_channels* height_image* width_image] | [batch_size X num_channels* height_out* width_out] | [batch_size X num_channels* height_image* width_image] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], pool_size=[height_pool, width_pool] | Computes the gradients wrt input of 2D maxpooling | -| bias_add | input, bias | [batch_size X num_channels* height_image* width_image] | [num_channels X 1] | [batch_size X num_channels* height_image* width_image] | | Adds the bias (row vector of size num_channels) to input with the given num_channels | -| bias_multiply | input, bias | [batch_size X num_channels* height_image* width_image] | [num_channels X 1] | [batch_size X num_channels* height_image* width_image] | | Multiplies the bias (row vector of size num_channels) to input with the given num_channels | +| Function name | Input matrices | Dimension of first input matrix | Dimension of second input matrix (if applicable) | Dimension of output matrix | Input Parameters | Notes | +|---------------------------------------------|----------------|-----------------------------------------------------------|-----------------------------------------------------------|------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------| +| conv2d | input, filter | [batch_size X num_channels* height_image* width_image] | [num_filters X num_channels* height_filter* width_filter] | [batch_size X num_channels_out* height_out* width_out] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], filter_shape=[num_filters, num_channels, height_filter, width_filter] | Performs 2D convolution operation | +| conv2d_backward_filter | input, dout | [batch_size X num_channels* height_image* width_image] | [batch_size X num_channels_out* height_out* width_out] | [num_filters X num_channels* height_filter* width_filter] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], filter_shape=[num_filters, num_channels, height_filter, width_filter] | Computes the gradients wrt filter of 2D convolution | +| conv2d_backward_data | filter, dout | [num_filters X num_channels* height_filter* width_filter] | [batch_size X num_channels_out* height_out* width_out] | [batch_size X num_channels* height_image* width_image] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], filter_shape=[num_filters, num_channels, height_filter, width_filter] | Computes the gradients wrt input of 2D convolution | +| max_pool, avg_pool | input | [batch_size X num_channels* height_image* width_image] | | [batch_size X num_channels* height_out* width_out] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], pool_size=[height_pool, width_pool] | Performs max/average pooling operation | +| max_pool_backward, avg_pool_backward | input, dout | [batch_size X num_channels* height_image* width_image] | [batch_size X num_channels* height_out* width_out] | [batch_size X num_channels* height_image* width_image] | stride=[stride_h, stride_w], padding=[pad_h, pad_w], input_shape=[batch_size, num_channels, height_image, width_image], pool_size=[height_pool, width_pool] | Computes the gradients wrt input of 2D max pooling, average pooling | +| bias_add | input, bias | [batch_size X num_channels* height_image* width_image] | [num_channels X 1] | [batch_size X num_channels* height_image* width_image] | | Adds the bias (row vector of size num_channels) to input with the given num_channels | +| bias_multiply | input, bias | [batch_size X num_channels* height_image* width_image] | [num_channels X 1] | [batch_size X num_channels* height_image* width_image] | | Multiplies the bias (row vector of size num_channels) to input with the given num_channels | Examples: http://git-wip-us.apache.org/repos/asf/systemml/blob/781fa43d/reference-guide-caffe2dml.md ---------------------------------------------------------------------- diff --git a/reference-guide-caffe2dml.md b/reference-guide-caffe2dml.md index 0e191dd..8e2ed1f 100644 --- a/reference-guide-caffe2dml.md +++ b/reference-guide-caffe2dml.md @@ -97,7 +97,7 @@ Invokes [nn/layers/max_pool2d_builtin.dml](https://github.com/apache/systemml/bl - kernel_size (or kernel_h and kernel_w): specifies height and width of each filter **Optional Parameters:** -- pool (default MAX): the pooling method. Currently, we only support MAX, not AVE, or STOCHASTIC. +- pool (default MAX): the pooling method. Currently, we only support MAX and AVE, not STOCHASTIC. - pad (or pad_h and pad_w) (default 0): specifies the number of pixels to (implicitly) add to each side of the input - stride (or stride_h and stride_w) (default 1): specifies the intervals at which to apply the filters to the input @@ -116,6 +116,30 @@ layer { } ``` + +### Upsampling Layer + +Invokes [nn/layers/upsample2d.dml](https://github.com/apache/systemml/blob/master/scripts/nn/layers/upsample2d.dml) layer. + +**Required Parameters:** + +- size_h and size_w: specifies the upsampling factor for rows and columns. + +**Sample Usage:** +``` +layer { + name: "upsample1" + type: "Upsample" + bottom: "pool1" + top: "upsample1" + upsample_param { + size_h = 2 + size_w = 2 + } +} +``` + + ### Deconvolution Layer Invokes [nn/layers/conv2d_transpose.dml](https://github.com/apache/systemml/blob/master/scripts/nn/layers/conv2d_transpose.dml)
