[GitHub] chaoyuaw commented on a change in pull request #9165: add embedding learning example
chaoyuaw commented on a change in pull request #9165: add embedding learning
example
URL: https://github.com/apache/incubator-mxnet/pull/9165#discussion_r166049971
##
File path: example/gluon/embedding_learning/model.py
##
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from mxnet import gluon
+from mxnet.gluon import nn, Block, HybridBlock
+import numpy as np
+
+class L2Normalization(HybridBlock):
+r"""Applies L2 Normalization to input.
+
+Parameters
+--
+mode : str
+Mode of normalization.
+See :func:`~mxnet.ndarray.L2Normalization` for available choices.
+
+Inputs:
+- **data**: input tensor with arbitrary shape.
+
+Outputs:
+- **out**: output tensor with the same shape as `data`.
+"""
+def __init__(self, mode, **kwargs):
+self._mode = mode
+super(L2Normalization, self).__init__(**kwargs)
+
+def hybrid_forward(self, F, x):
+return F.L2Normalization(x, mode=self._mode, name='l2_norm')
+
+def __repr__(self):
+s = '{name}({_mode})'
+return s.format(name=self.__class__.__name__,
+**self.__dict__)
+
+
+def get_distance(F, x):
+"""Helper function for margin-based loss. Return a distance matrix given a
matrix."""
+n = x.shape[0]
+
+square = F.sum(x ** 2.0, axis=1, keepdims=True)
+distance_square = square + square.transpose() - (2.0 * F.dot(x,
x.transpose()))
+
+# Adding identity to make sqrt work.
+return F.sqrt(distance_square + F.array(np.identity(n)))
+
+class DistanceWeightedSampling(HybridBlock):
+r"""Distance weighted sampling. See "sampling matters in deep embedding
learning"
+paper for details.
+
+Parameters
+--
+batch_k : int
+Number of images per class.
+
+Inputs:
+- **data**: input tensor with shape (batch_size, embed_dim).
+Here we assume the consecutive batch_k examples are of the same class.
+For example, if batch_k = 5, the first 5 examples belong to the same
class,
+6th-10th examples belong to another class, etc.
+
+Outputs:
+- a_indices: indices of anchors.
+- x[a_indices]: sampled anchor embeddings.
+- x[p_indices]: sampled positive embeddings.
+- x[n_indices]: sampled negative embeddings.
+- x: embeddings of the input batch.
+"""
+def __init__(self, batch_k, cutoff=0.5, nonzero_loss_cutoff=1.4, **kwargs):
+self.batch_k = batch_k
+self.cutoff = cutoff
+
+# We sample only from negatives that induce a non-zero loss.
+# These are negatives with a distance < nonzero_loss_cutoff.
+# With a margin-based loss, nonzero_loss_cutoff == margin + beta.
+self.nonzero_loss_cutoff = nonzero_loss_cutoff
+super(DistanceWeightedSampling, self).__init__(**kwargs)
+
+def hybrid_forward(self, F, x):
+k = self.batch_k
+n, d = x.shape
+
+distance = get_distance(F, x)
+# Cut off to avoid high variance.
+distance = F.maximum(distance, self.cutoff)
+
+# Subtract max(log(distance)) for stability.
+log_weights = ((2.0 - float(d)) * F.log(distance)
+ - (float(d - 3) / 2) * F.log(1.0 - 0.25 * (distance **
2.0)))
+weights = F.exp(log_weights - F.max(log_weights))
+
+# Sample only negative examples by setting weights of
+# the same-class examples to 0.
+mask = np.ones(weights.shape)
+for i in range(0, n, k):
+mask[i:i+k, i:i+k] = 0
+
+weights = weights * F.array(mask) * (distance <
self.nonzero_loss_cutoff)
+weights = weights / F.sum(weights, axis=1, keepdims=True)
+
+a_indices = []
+p_indices = []
+n_indices = []
+
+np_weights = weights.asnumpy()
+for i in range(n):
+block_idx = i // k
+
+try:
+n_indices += np.random.choice(n, k-1, p=np_weights[i]).tolist()
+except:
+n_indices += np.random.choice(n, k-1).tolist()
+for j in range(block_idx * k, (block_idx + 1) * k):
+if j != i:
+
[GitHub] chaoyuaw commented on a change in pull request #9165: add embedding learning example
chaoyuaw commented on a change in pull request #9165: add embedding learning
example
URL: https://github.com/apache/incubator-mxnet/pull/9165#discussion_r165834433
##
File path: example/gluon/embedding_learning/model.py
##
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from mxnet import gluon
+from mxnet.gluon import nn, Block, HybridBlock
+import numpy as np
+
+class L2Normalization(HybridBlock):
+r"""Applies L2 Normalization to input.
+
+Parameters
+--
+mode : str
+Mode of normalization.
+See :func:`~mxnet.ndarray.L2Normalization` for available choices.
+
+Inputs:
+- **data**: input tensor with arbitrary shape.
+
+Outputs:
+- **out**: output tensor with the same shape as `data`.
+"""
+def __init__(self, mode, **kwargs):
+self._mode = mode
+super(L2Normalization, self).__init__(**kwargs)
+
+def hybrid_forward(self, F, x):
+return F.L2Normalization(x, mode=self._mode, name='l2_norm')
+
+def __repr__(self):
+s = '{name}({_mode})'
+return s.format(name=self.__class__.__name__,
+**self.__dict__)
+
+
+def get_distance(F, x):
+"""Helper function for margin-based loss. Return a distance matrix given a
matrix."""
+n = x.shape[0]
+
+square = F.sum(x ** 2.0, axis=1, keepdims=True)
+distance_square = square + square.transpose() - (2.0 * F.dot(x,
x.transpose()))
+
+# Adding identity to make sqrt work.
+return F.sqrt(distance_square + F.array(np.identity(n)))
+
+class DistanceWeightedSampling(HybridBlock):
+r"""Distance weighted sampling. See "sampling matters in deep embedding
learning"
+paper for details.
+
+Parameters
+--
+batch_k : int
+Number of images per class.
+
+Inputs:
+- **data**: input tensor with shape (batch_size, embed_dim).
+Here we assume the consecutive batch_k examples are of the same class.
+For example, if batch_k = 5, the first 5 examples belong to the same
class,
+6th-10th examples belong to another class, etc.
+
+Outputs:
+- a_indices: indices of anchors.
+- x[a_indices]: sampled anchor embeddings.
+- x[p_indices]: sampled positive embeddings.
+- x[n_indices]: sampled negative embeddings.
+- x: embeddings of the input batch.
+"""
+def __init__(self, batch_k, cutoff=0.5, nonzero_loss_cutoff=1.4, **kwargs):
+self.batch_k = batch_k
+self.cutoff = cutoff
+
+# We sample only from negatives that induce a non-zero loss.
+# These are negatives with a distance < nonzero_loss_cutoff.
+# With a margin-based loss, nonzero_loss_cutoff == margin + beta.
+self.nonzero_loss_cutoff = nonzero_loss_cutoff
+super(DistanceWeightedSampling, self).__init__(**kwargs)
+
+def hybrid_forward(self, F, x):
+k = self.batch_k
+n, d = x.shape
+
+distance = get_distance(F, x)
+# Cut off to avoid high variance.
+distance = F.maximum(distance, self.cutoff)
+
+# Subtract max(log(distance)) for stability.
+log_weights = ((2.0 - float(d)) * F.log(distance)
+ - (float(d - 3) / 2) * F.log(1.0 - 0.25 * (distance **
2.0)))
+weights = F.exp(log_weights - F.max(log_weights))
+
+# Sample only negative examples by setting weights of
+# the same-class examples to 0.
+mask = np.ones(weights.shape)
+for i in range(0, n, k):
+mask[i:i+k, i:i+k] = 0
+
+weights = weights * F.array(mask) * (distance <
self.nonzero_loss_cutoff)
+weights = weights / F.sum(weights, axis=1, keepdims=True)
+
+a_indices = []
+p_indices = []
+n_indices = []
+
+np_weights = weights.asnumpy()
+for i in range(n):
+block_idx = i // k
+
+try:
+n_indices += np.random.choice(n, k-1, p=np_weights[i]).tolist()
+except:
+n_indices += np.random.choice(n, k-1).tolist()
Review comment:
Thanks a lot for pointing this out! Yes, you're absolutely right. I'll fix
this soon.
[GitHub] chaoyuaw commented on a change in pull request #9165: add embedding learning example
chaoyuaw commented on a change in pull request #9165: add embedding learning example URL: https://github.com/apache/incubator-mxnet/pull/9165#discussion_r165834384 ## File path: example/gluon/embedding_learning/README.md ## @@ -0,0 +1,72 @@ +# Image Embedding Learning + +This example implements embedding learning based on a Margin-based Loss with distance weighted sampling [(Wu et al, 2017)](http://www.philkr.net/papers/2017-10-01-iccv/2017-10-01-iccv.pdf). The model obtains a validation Recall@1 of ~64% on the [Caltech-UCSD Birds-200-2011](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html) dataset. Review comment: Many thanks for the good suggestion! Yes, I will update this soon. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chaoyuaw commented on a change in pull request #9165: add embedding learning example
chaoyuaw commented on a change in pull request #9165: add embedding learning example URL: https://github.com/apache/incubator-mxnet/pull/9165#discussion_r159566297 ## File path: example/gluon/embedding_learning/README.md ## @@ -0,0 +1,72 @@ +# Image Embedding Learning + +This example implements embedding learning based on a Margin-based Loss with distance weighted sampling [(Wu et al, 2017)](http://www.philkr.net/papers/2017-10-01-iccv/2017-10-01-iccv.pdf). The model obtains a validation Recall@1 of ~64% on the [Caltech-UCSD Birds-200-2011](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html) dataset. Review comment: Yes, approximately (slightly higher recall@1 and slightly lower recall@16 than what's reported in the paper). The difference is < 1%. The difference between this implementation and the original implementation is that here we perform sampling within each GPU while the original paper implements cross-gpu sampling. Since the performance is almost identical (at least on this dataset), I use per-gpu sampling here for simplicity. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
