ivandika3 commented on code in PR #4696:
URL: https://github.com/apache/ozone/pull/4696#discussion_r1190872522
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperTask.java:
##########
@@ -383,4 +369,55 @@ private void handlePutOMKeyEvent(String key, OmKeyInfo
omKeyInfo,
}
}
+ /**
+ * Write an OM key to container DB and update containerID -> no. of keys
+ * count.
+ *
+ * @param key key String
+ * @param omKeyInfo omKeyInfo value
+ * @throws IOException if unable to write to recon DB.
+ */
+ private void writeOMKeyToContainerDB(String key, OmKeyInfo omKeyInfo)
+ throws IOException {
+ long containerCountToIncrement = 0;
+ for (OmKeyLocationInfoGroup omKeyLocationInfoGroup : omKeyInfo
+ .getKeyLocationVersions()) {
+ long keyVersion = omKeyLocationInfoGroup.getVersion();
+ for (OmKeyLocationInfo omKeyLocationInfo : omKeyLocationInfoGroup
+ .getLocationList()) {
+ long containerId = omKeyLocationInfo.getContainerID();
+ ContainerKeyPrefix containerKeyPrefix = new ContainerKeyPrefix(
+ containerId, key, keyVersion);
+ if (reconContainerMetadataManager.getCountForContainerKeyPrefix(
+ containerKeyPrefix) == 0) {
+ // Save on writes. No need to save same container-key prefix
+ // mapping again.
+ reconContainerMetadataManager.storeContainerKeyMapping(
+ containerKeyPrefix, 1);
+
+ // check if container already exists and
+ // increment the count of containers if it does not exist
+ if (!reconContainerMetadataManager.doesContainerExists(containerId))
{
+ containerCountToIncrement++;
+ }
+
+ // update the count of keys for the given containerID
+ long keyCount = reconContainerMetadataManager
+ .getKeyCountForContainer(containerId);
Review Comment:
Following the previous conversation, we can follow the implementation in
`handlePutOMKeyEvent` function using the `containerKeyCountMap` instead, as
follows:
```
// update the count of keys for the given containerID
long keyCount;
if (containerKeyCountMap.containsKey(containerId)) {
keyCount = containerKeyCountMap.get(containerId);
} else {
containerCountToIncrement++;
keyCount = 0;
}
// increment the count and update containerKeyCount.
// keyCount will be 0 if containerID is not found. So, there is no
// need to initialize keyCount for the first time.
containerKeyCountMap.put(containerId, ++keyCount);
```
If the container has been encountered before, we can use the look up from
`containerKeyCountMap`.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperTask.java:
##########
@@ -383,4 +369,55 @@ private void handlePutOMKeyEvent(String key, OmKeyInfo
omKeyInfo,
}
}
+ /**
+ * Write an OM key to container DB and update containerID -> no. of keys
+ * count.
+ *
+ * @param key key String
+ * @param omKeyInfo omKeyInfo value
+ * @throws IOException if unable to write to recon DB.
+ */
+ private void writeOMKeyToContainerDB(String key, OmKeyInfo omKeyInfo)
+ throws IOException {
+ long containerCountToIncrement = 0;
+ for (OmKeyLocationInfoGroup omKeyLocationInfoGroup : omKeyInfo
+ .getKeyLocationVersions()) {
+ long keyVersion = omKeyLocationInfoGroup.getVersion();
+ for (OmKeyLocationInfo omKeyLocationInfo : omKeyLocationInfoGroup
+ .getLocationList()) {
+ long containerId = omKeyLocationInfo.getContainerID();
+ ContainerKeyPrefix containerKeyPrefix = new ContainerKeyPrefix(
+ containerId, key, keyVersion);
+ if (reconContainerMetadataManager.getCountForContainerKeyPrefix(
+ containerKeyPrefix) == 0) {
+ // Save on writes. No need to save same container-key prefix
+ // mapping again.
+ reconContainerMetadataManager.storeContainerKeyMapping(
+ containerKeyPrefix, 1);
+
+ // check if container already exists and
+ // increment the count of containers if it does not exist
+ if (!reconContainerMetadataManager.doesContainerExists(containerId))
{
+ containerCountToIncrement++;
Review Comment:
Using `containerKeyCountMap`:
```
if (!containerKeyCountMap.containsKey(containerId)) {
containerCountToIncrement++;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
