errose28 commented on code in PR #3286:
URL: https://github.com/apache/ozone/pull/3286#discussion_r866283268
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java:
##########
@@ -196,36 +196,41 @@ public OMClientResponse
validateAndUpdateCache(OzoneManager ozoneManager,
}
}
+ // If bucket versioning is turned on during the update, between key
+ // creation and key commit, old versions will be just overwritten and
+ // not kept. Bucket versioning will be effective from the first key
+ // creation after the knob turned on.
+ RepeatedOmKeyInfo oldKeyVersionsToDelete = null;
+ OmKeyInfo keyToDelete =
+ omMetadataManager.getKeyTable(getBucketLayout()).get(dbOzoneKey);
+ if (keyToDelete != null && !omBucketInfo.getIsVersionEnabled()) {
+ oldKeyVersionsToDelete = getOldVersionsToCleanUp(dbOzoneKey,
+ keyToDelete, omMetadataManager,
+ trxnLogIndex, ozoneManager.isRatisEnabled());
+ }
+
omKeyInfo =
omMetadataManager.getOpenKeyTable(getBucketLayout()).get(dbOpenKey);
if (omKeyInfo == null) {
throw new OMException("Failed to commit key, as " + dbOpenKey +
"entry is not found in the OpenKey table", KEY_NOT_FOUND);
}
+ omBucketInfo = getBucketInfo(omMetadataManager, volumeName, bucketName);
omKeyInfo.setDataSize(commitKeyArgs.getDataSize());
-
omKeyInfo.setModificationTime(commitKeyArgs.getModificationTime());
-
// Update the block length for each block
- List<OmKeyLocationInfo> allocatedLocationInfoList =
- omKeyInfo.getLatestVersionLocations().getLocationList();
omKeyInfo.updateLocationInfoList(locationInfoList, false);
-
// Set the UpdateID to current transactionLogIndex
omKeyInfo.setUpdateID(trxnLogIndex, ozoneManager.isRatisEnabled());
- // If bucket versioning is turned on during the update, between key
- // creation and key commit, old versions will be just overwritten and
- // not kept. Bucket versioning will be effective from the first key
- // creation after the knob turned on.
- RepeatedOmKeyInfo oldKeyVersionsToDelete = null;
- OmKeyInfo keyToDelete =
- omMetadataManager.getKeyTable(getBucketLayout()).get(dbOzoneKey);
- if (keyToDelete != null && !omBucketInfo.getIsVersionEnabled()) {
- oldKeyVersionsToDelete = getOldVersionsToCleanUp(dbOzoneKey,
- keyToDelete, omMetadataManager,
- trxnLogIndex, ozoneManager.isRatisEnabled());
+ long correctedSpace = omKeyInfo.getReplicatedSize();
+ // Subtract the size of blocks to be overwritten.
+ if (keyToDelete != null) {
+ correctedSpace -= keyToDelete.getReplicatedSize();
Review Comment:
@captainzmc I don't totally follow the logic in this part, and the
corresponding part in OMKeyCommitRequestWithFSO. Could you clarify how this is
handling quota calculations when we are/are not overwriting a key, and how
versioning be enabled or disabled affects this?
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMAllocateBlockRequestWithFSO.java:
##########
@@ -144,7 +144,12 @@ public OMClientResponse
validateAndUpdateCache(OzoneManager ozoneManager,
long preAllocatedSpace = newLocationList.size()
* ozoneManager.getScmBlockSize()
* openKeyInfo.getReplicationConfig().getRequiredNodes();
- checkBucketQuotaInBytes(omBucketInfo, preAllocatedSpace);
+ long hadAllocatedSpace =
+ openKeyInfo.getLatestVersionLocations().getLocationList().size()
+ * ozoneManager.getScmBlockSize()
+ * openKeyInfo.getReplicationConfig().getRequiredNodes();
Review Comment:
I don't think this will give the correct result for EC. Looks like
`QuotaUtil#getReplicatedSize` has logic that might help here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
