[GitHub] haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling
haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling
and Global Lp Pooling
URL: https://github.com/apache/incubator-mxnet/pull/10780#discussion_r189058395
##
File path: tests/python/gpu/test_operator_gpu.py
##
@@ -769,140 +769,161 @@ def test_pooling_versions_helper(pool_op_list, data,
kernel, pool_type, pad, str
ctx_list.append({'ctx': mx.cpu(0), 'pool_data': data, 'type_dict':
{'pool_data': np.float32}})
if not global_pool:
sym_list.append(mx.sym.Pooling(kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
-
pooling_convention=pooling_convention, name='pool'))
+
pooling_convention=pooling_convention, name='pool', p_value=p_value))
else:
-sym_list.append(mx.sym.Pooling(kernel=kernel,
pool_type=pool_type, global_pool=True, name='pool'))
+sym_list.append(mx.sym.Pooling(kernel=kernel,
pool_type=pool_type, global_pool=True, name='pool', p_value=p_value))
# Pooling gpu
if 'pool_gpu' in pool_op_list:
ctx_list.append({'ctx': mx.gpu(0), 'pool_data': data, 'type_dict':
{'pool_data': np.float32}})
if not global_pool:
sym_list.append(mx.sym.Pooling(kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
-
pooling_convention=pooling_convention, cudnn_off=True, name='pool'))
+
pooling_convention=pooling_convention, cudnn_off=True, name='pool',
p_value=p_value))
else:
sym_list.append(mx.sym.Pooling(kernel=kernel,
pool_type=pool_type, global_pool=True, cudnn_off=True,
- name='pool'))
+ name='pool', p_value=p_value))
# CuDNNPooling
if 'pool_cudnn' in pool_op_list:
ctx_list.append({'ctx': mx.gpu(0), 'pool_data': data, 'type_dict':
{'pool_data': np.float32}})
if not global_pool:
sym_list.append(mx.sym.Pooling(kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
-
pooling_convention=pooling_convention, cudnn_off=False, name='pool'))
+
pooling_convention=pooling_convention, p_value=p_value, cudnn_off=False,
name='pool'))
else:
-sym_list.append(mx.sym.Pooling(kernel=kernel,
pool_type=pool_type, global_pool=True, cudnn_off=False,
- name='pool'))
+sym_list.append(mx.sym.Pooling(kernel=kernel,
pool_type=pool_type, global_pool=True, p_value=p_value,
+ cudnn_off=False, name='pool'))
check_consistency(sym_list, ctx_list)
-def test_1d_pooling(pool_type):
+def test_1d_pooling(pool_type, p_value=2):
data = (2, 3, 20)
kernel = (4,)
pad = (0,)
stride = (1,)
test_pooling_versions_helper(pool_op_list=['pool_cpu', 'pool_gpu'],
data=data, kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
- pooling_convention='valid',
global_pool=False)
+ pooling_convention='valid',
global_pool=False, p_value=p_value)
pad = (2,)
stride = (2,)
test_pooling_versions_helper(pool_op_list=['pool_cpu', 'pool_gpu'],
data=data, kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
- pooling_convention='valid',
global_pool=False)
+ pooling_convention='valid',
global_pool=False, p_value=p_value)
pad = (0,)
stride = (1,)
test_pooling_versions_helper(pool_op_list=['pool_cpu', 'pool_gpu'],
data=data, kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
- pooling_convention='full',
global_pool=False)
+ pooling_convention='full',
global_pool=False, p_value=p_value)
pad = (2,)
stride = (2,)
test_pooling_versions_helper(pool_op_list=['pool_cpu', 'pool_gpu'],
data=data, kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
- pooling_convention='full',
global_pool=False)
+ pooling_convention='full',
global_pool=False, p_value=p_value)
test_pooling_versions_helper(pool_op_list=['pool_cpu', 'pool_gpu'],
data=data, kernel=kernel, pad=pad,
stride=stride, pool_type=pool_type,
-
[GitHub] haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling
haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling
and Global Lp Pooling
URL: https://github.com/apache/incubator-mxnet/pull/10780#discussion_r189051853
##
File path: src/operator/nn/pooling.cc
##
@@ -92,6 +92,9 @@ static bool PoolingShape(const nnvm::NodeAttrs &attrs,
std::vector *out_shape) {
const PoolingParam ¶m = nnvm::get(attrs.parsed);
CHECK_EQ(in_shape->size(), 1U);
+ if (param.pool_type == pool_enum::kLpPooling) {
+CHECK(param.p_value.has_value());
Review comment:
We'd better not do this as this may affect Jun's work. If we set p_value to
be defaulting to 1 instead of optional, the generated json file for this symbol
would contain an extra field.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling
haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling URL: https://github.com/apache/incubator-mxnet/pull/10780#discussion_r187409574 ## File path: src/operator/nn/pooling.cc ## @@ -344,11 +347,18 @@ Three pooling options are supported by ``pool_type``: - **avg**: average pooling - **max**: max pooling - **sum**: sum pooling +- **lp**: Lp pooling For 3-D pooling, an additional *depth* dimension is added before *height*. Namely the input data will have shape *(batch_size, channel, depth, height, width)*. +Notes on Lp pooling: + +Lp pooling was first introduced by this paper: https://arxiv.org/pdf/1204.3968.pdf. Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling
haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling
and Global Lp Pooling
URL: https://github.com/apache/incubator-mxnet/pull/10780#discussion_r187247799
##
File path: src/operator/nn/pool_utils.h
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#ifndef MXNET_OPERATOR_NN_POOL_UTILS_H_
+#define MXNET_OPERATOR_NN_POOL_UTILS_H_
+
+#include "../mshadow_op.h"
+
+namespace mxnet {
+namespace op {
+
+template
+struct a_to_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return mshadow_op::power::Map(a, DType(p));
+ }
+};
+
+template
+struct a_to_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return a;
+ }
+};
+
+template
+struct a_to_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return a*a;
+ }
+};
+
+template
+struct a_to_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return a*a*a;
+ }
+};
+
+template
+struct a_root_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return mshadow_op::power::Map(a, DType(1.0 / p));
+ }
+};
+
+template
+struct a_root_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return a;
+ }
+};
+
+template
+struct a_root_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return mshadow_op::square_root::Map(a);
+ }
+};
+
+template
+struct a_root_p {
+ static MSHADOW_XINLINE DType Map(DType a) {
+return mshadow_op::cube_root::Map(a);
+ }
+};
+
+template
+struct lp_grad {
+ static MSHADOW_XINLINE DType Map(DType grad, DType in_data, DType out_data) {
+return grad * mshadow_op::power::Map(in_data / out_data, DType(p - 1));
+ }
+};
+
+template
+struct lp_grad {
+ static MSHADOW_XINLINE DType Map(DType grad, DType in_data, DType out_data) {
+return grad;
+ }
+};
+
+template
+struct lp_grad {
+ static MSHADOW_XINLINE DType Map(DType grad, DType in_data, DType out_data) {
+return grad * in_data / out_data;
+ }
+};
+
+template
+struct lp_grad {
+ static MSHADOW_XINLINE DType Map(DType grad, DType in_data, DType out_data) {
+return grad * in_data * in_data / out_data / out_data;
Review comment:
Do you mean something like ```return grad * in_data * in_data / (out_data *
out_data)```?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling
haojin2 commented on a change in pull request #10780: [MXNET-375] Lp Pooling and Global Lp Pooling URL: https://github.com/apache/incubator-mxnet/pull/10780#discussion_r186846020 ## File path: tests/python/gpu/test_operator_gpu.py ## @@ -745,7 +745,7 @@ def test_pooling_with_type(): @with_seed() def test_pooling_versions(): def test_pooling_versions_helper(pool_op_list, data, kernel, pool_type, pad, stride, - pooling_convention='valid', global_pool=False): + pooling_convention='valid', global_pool=False, p_value=2): Review comment: The most commonly used case for Lp pooling is p=2, for all other types of pooling this value is not used at all, so there's no side effect on other types of pooling with setting p_value to be 2. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
