[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r172035107
##
File path: src/operator/contrib/proposal.cu
##
@@ -553,10 +553,10 @@ class ProposalGPUOp : public Operator{
cudaMemcpyHostToDevice));
// copy results after nms
-dimGrid.x = (rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
+dimGrid.x = (param_.rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
Review comment:
Thank you!
The invalid anchors would be ranked to the bottom for NMS, but the threshold
is for overlap between two anchors rather than score.
Should we add a condition that `score != -1` in NMS?
https://github.com/apache/incubator-mxnet/blob/master/src/operator/contrib/proposal.cc#L259
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r172003236
##
File path: src/operator/contrib/proposal.cu
##
@@ -553,10 +553,10 @@ class ProposalGPUOp : public Operator{
cudaMemcpyHostToDevice));
// copy results after nms
-dimGrid.x = (rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
+dimGrid.x = (param_.rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
Review comment:
@precedenceguo Thank you!
I have a question.
The score of the anchor boxes whose size are less than `rpn_min_size` will
be assigned to -1 (FilterBox), then these anchor boxes may be NMS and selected
as the output in the implementation. Are they valid anchors?
I think they are invalid and should be thrown away.
Reference:
https://github.com/precedenceguo/mx-rcnn/blob/master/rcnn/symbol/proposal.py#L116
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r172003236
##
File path: src/operator/contrib/proposal.cu
##
@@ -553,10 +553,10 @@ class ProposalGPUOp : public Operator{
cudaMemcpyHostToDevice));
// copy results after nms
-dimGrid.x = (rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
+dimGrid.x = (param_.rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
Review comment:
@precedenceguo Thank you!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r171876138
##
File path: src/operator/contrib/multi_proposal.cc
##
@@ -22,11 +22,253 @@
* Licensed under The Apache-2.0 License [see LICENSE for details]
* \file multi_proposal.cc
* \brief
- * \author Xizhou Zhu
+ * \author Xizhou Zhu, Kan Wu
*/
#include "./multi_proposal-inl.h"
+//
+// Bounding Box Transform Utils
+//
+namespace mxnet {
+namespace op {
+namespace utils {
+
+// bbox prediction and clip to the image borders
+inline void BBoxTransformInv(const mshadow::Tensor& boxes,
+ const mshadow::Tensor& deltas,
+ const float im_height,
+ const float im_width,
+ const int real_height,
+ const int real_width,
+ mshadow::Tensor *out_pred_boxes) {
+ CHECK_GE(boxes.size(1), 4);
+ CHECK_GE(out_pred_boxes->size(1), 4);
+ int anchors = deltas.size(0) / 4;
+ int heights = deltas.size(1);
+ int widths = deltas.size(2);
+
+ for (int a = 0; a < anchors; ++a) {
+for (int h = 0; h < heights; ++h) {
+ for (int w = 0; w < widths; ++w) {
+index_t index = h * (widths * anchors) + w * (anchors) + a;
+float width = boxes[index][2] - boxes[index][0] + 1.0;
+float height = boxes[index][3] - boxes[index][1] + 1.0;
+float ctr_x = boxes[index][0] + 0.5 * (width - 1.0);
+float ctr_y = boxes[index][1] + 0.5 * (height - 1.0);
+
+float dx = deltas[a*4 + 0][h][w];
+float dy = deltas[a*4 + 1][h][w];
+float dw = deltas[a*4 + 2][h][w];
+float dh = deltas[a*4 + 3][h][w];
+
+float pred_ctr_x = dx * width + ctr_x;
+float pred_ctr_y = dy * height + ctr_y;
+float pred_w = exp(dw) * width;
+float pred_h = exp(dh) * height;
+
+float pred_x1 = pred_ctr_x - 0.5 * (pred_w - 1.0);
+float pred_y1 = pred_ctr_y - 0.5 * (pred_h - 1.0);
+float pred_x2 = pred_ctr_x + 0.5 * (pred_w - 1.0);
+float pred_y2 = pred_ctr_y + 0.5 * (pred_h - 1.0);
+
+pred_x1 = std::max(std::min(pred_x1, im_width - 1.0f), 0.0f);
+pred_y1 = std::max(std::min(pred_y1, im_height - 1.0f), 0.0f);
+pred_x2 = std::max(std::min(pred_x2, im_width - 1.0f), 0.0f);
+pred_y2 = std::max(std::min(pred_y2, im_height - 1.0f), 0.0f);
+
+(*out_pred_boxes)[index][0] = pred_x1;
+(*out_pred_boxes)[index][1] = pred_y1;
+(*out_pred_boxes)[index][2] = pred_x2;
+(*out_pred_boxes)[index][3] = pred_y2;
+
+if (h >= real_height || w >= real_width) {
+ (*out_pred_boxes)[index][4] = -1.0;
+}
+ }
+}
+ }
+}
+
+// iou prediction and clip to the image border
+inline void IoUTransformInv(const mshadow::Tensor& boxes,
+const mshadow::Tensor& deltas,
+const float im_height,
+const float im_width,
+const int real_height,
+const int real_width,
+mshadow::Tensor *out_pred_boxes) {
+ CHECK_GE(boxes.size(1), 4);
+ CHECK_GE(out_pred_boxes->size(1), 4);
+ int anchors = deltas.size(0) / 4;
+ int heights = deltas.size(1);
+ int widths = deltas.size(2);
+
+ for (int a = 0; a < anchors; ++a) {
+for (int h = 0; h < heights; ++h) {
+ for (int w = 0; w < widths; ++w) {
+index_t index = h * (widths * anchors) + w * (anchors) + a;
Review comment:
@pengzhao-intel Thank you:) I will read the document and try it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r171778623
##
File path: src/operator/contrib/multi_proposal.cc
##
@@ -22,11 +22,253 @@
* Licensed under The Apache-2.0 License [see LICENSE for details]
* \file multi_proposal.cc
* \brief
- * \author Xizhou Zhu
+ * \author Xizhou Zhu, Kan Wu
*/
#include "./multi_proposal-inl.h"
+//
+// Bounding Box Transform Utils
+//
+namespace mxnet {
+namespace op {
+namespace utils {
+
+// bbox prediction and clip to the image borders
+inline void BBoxTransformInv(const mshadow::Tensor& boxes,
+ const mshadow::Tensor& deltas,
+ const float im_height,
+ const float im_width,
+ const int real_height,
+ const int real_width,
+ mshadow::Tensor *out_pred_boxes) {
+ CHECK_GE(boxes.size(1), 4);
+ CHECK_GE(out_pred_boxes->size(1), 4);
+ int anchors = deltas.size(0) / 4;
+ int heights = deltas.size(1);
+ int widths = deltas.size(2);
+
+ for (int a = 0; a < anchors; ++a) {
+for (int h = 0; h < heights; ++h) {
+ for (int w = 0; w < widths; ++w) {
+index_t index = h * (widths * anchors) + w * (anchors) + a;
+float width = boxes[index][2] - boxes[index][0] + 1.0;
+float height = boxes[index][3] - boxes[index][1] + 1.0;
+float ctr_x = boxes[index][0] + 0.5 * (width - 1.0);
+float ctr_y = boxes[index][1] + 0.5 * (height - 1.0);
+
+float dx = deltas[a*4 + 0][h][w];
+float dy = deltas[a*4 + 1][h][w];
+float dw = deltas[a*4 + 2][h][w];
+float dh = deltas[a*4 + 3][h][w];
+
+float pred_ctr_x = dx * width + ctr_x;
+float pred_ctr_y = dy * height + ctr_y;
+float pred_w = exp(dw) * width;
+float pred_h = exp(dh) * height;
+
+float pred_x1 = pred_ctr_x - 0.5 * (pred_w - 1.0);
+float pred_y1 = pred_ctr_y - 0.5 * (pred_h - 1.0);
+float pred_x2 = pred_ctr_x + 0.5 * (pred_w - 1.0);
+float pred_y2 = pred_ctr_y + 0.5 * (pred_h - 1.0);
+
+pred_x1 = std::max(std::min(pred_x1, im_width - 1.0f), 0.0f);
+pred_y1 = std::max(std::min(pred_y1, im_height - 1.0f), 0.0f);
+pred_x2 = std::max(std::min(pred_x2, im_width - 1.0f), 0.0f);
+pred_y2 = std::max(std::min(pred_y2, im_height - 1.0f), 0.0f);
+
+(*out_pred_boxes)[index][0] = pred_x1;
+(*out_pred_boxes)[index][1] = pred_y1;
+(*out_pred_boxes)[index][2] = pred_x2;
+(*out_pred_boxes)[index][3] = pred_y2;
+
+if (h >= real_height || w >= real_width) {
+ (*out_pred_boxes)[index][4] = -1.0;
+}
+ }
+}
+ }
+}
+
+// iou prediction and clip to the image border
+inline void IoUTransformInv(const mshadow::Tensor& boxes,
+const mshadow::Tensor& deltas,
+const float im_height,
+const float im_width,
+const int real_height,
+const int real_width,
+mshadow::Tensor *out_pred_boxes) {
+ CHECK_GE(boxes.size(1), 4);
+ CHECK_GE(out_pred_boxes->size(1), 4);
+ int anchors = deltas.size(0) / 4;
+ int heights = deltas.size(1);
+ int widths = deltas.size(2);
+
+ for (int a = 0; a < anchors; ++a) {
+for (int h = 0; h < heights; ++h) {
+ for (int w = 0; w < widths; ++w) {
+index_t index = h * (widths * anchors) + w * (anchors) + a;
Review comment:
@cjolivier01 I think we can merge the three for-loop to a for-loop, then use
OMP to optimize it
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r171544927
##
File path: src/operator/contrib/proposal.cu
##
@@ -553,10 +553,10 @@ class ProposalGPUOp : public Operator{
cudaMemcpyHostToDevice));
// copy results after nms
-dimGrid.x = (rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
+dimGrid.x = (param_.rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
Review comment:
@ZiyueHuang If `count_anchor` < `param_.rpn_post_nms_top_n`,
`rpn_post_nms_top_n` will be equal to `count_anchor`,
and `rpn_post_nms_top_n` will be less than `param_.rpn_post_nms_top_n`.
https://github.com/apache/incubator-mxnet/blob/master/src/operator/contrib/proposal-inl.h#L119
It shows the `batch_size` of the output is `param_.rpn_post_nms_top_n`
instead of `rpn_post_nms_top_n`.
The original version will not assign the element whose index is larger than
`rpn_post_nms_top_n`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r171544927
##
File path: src/operator/contrib/proposal.cu
##
@@ -553,10 +553,10 @@ class ProposalGPUOp : public Operator{
cudaMemcpyHostToDevice));
// copy results after nms
-dimGrid.x = (rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
+dimGrid.x = (param_.rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
Review comment:
@ZiyueHuang If `count_anchor` < `param_.rpn_post_nms_top_n`,
`rpn_post_nms_top_n` will be equal to `count_anchor`,
and `rpn_post_nms_top_n` will be less than `param_.rpn_post_nms_top_n`.
https://github.com/apache/incubator-mxnet/blob/master/src/operator/contrib/proposal-inl.h#L119
It shows the `batch_size` of the output is `param_.rpn_post_nms_top_n`
instead of `rpn_post_nms_top_n`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wkcn commented on a change in pull request #9939: add multi proposal operator (cpu version) and fix the bug in proposal op (gpu version)
wkcn commented on a change in pull request #9939: add multi proposal operator
(cpu version) and fix the bug in proposal op (gpu version)
URL: https://github.com/apache/incubator-mxnet/pull/9939#discussion_r171544927
##
File path: src/operator/contrib/proposal.cu
##
@@ -553,10 +553,10 @@ class ProposalGPUOp : public Operator{
cudaMemcpyHostToDevice));
// copy results after nms
-dimGrid.x = (rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
+dimGrid.x = (param_.rpn_post_nms_top_n + kMaxThreadsPerBlock - 1) /
kMaxThreadsPerBlock;
Review comment:
@ZiyueHuang If `count_anchor` < `param_.rpn_post_nms_top_n`,
`rpn_post_nms_top_n` will be `count_anchor`,
and `rpn_post_nms_top_n` will be less than `param_.rpn_post_nms_top_n`.
https://github.com/apache/incubator-mxnet/blob/master/src/operator/contrib/proposal-inl.h#L119
It shows the `batch_size` of the output is `param_.rpn_post_nms_top_n`
instead of `rpn_post_nms_top_n`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
