bharatviswa504 commented on a change in pull request #1679:
URL: https://github.com/apache/ozone/pull/1679#discussion_r541135722
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java
##########
@@ -556,9 +560,10 @@ public static OmKeyInfo getOmKeyInfoFromFileTable(boolean
openFileTable,
// For example, the user given key path is '/a/b/c/d/e/file1', then in DB
// keyName field stores only the leaf node name, which is 'file1'.
if (dbOmKeyInfo != null) {
- dbOmKeyInfo.setKeyName(keyName);
Review comment:
Why do we need this copyObject here?
Because get() always returns new object. (if it exists in cache, it returns
copyObject, if in DB, get new parsed Object from bytes.
```
public VALUE get(KEY key) throws IOException {
// Here the metadata lock will guarantee that cache is not updated for
same
// key during get key.
CacheResult<CacheValue<VALUE>> cacheResult =
cache.lookup(new CacheKey<>(key));
if (cacheResult.getCacheStatus() == EXISTS) {
return codecRegistry.copyObject(cacheResult.getValue().getCacheValue(),
valueType);
} else if (cacheResult.getCacheStatus() == NOT_EXIST) {
return null;
} else {
return getFromTable(key);
}
}
```
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java
##########
@@ -851,4 +856,72 @@ private static boolean checkSubFileExists(OmKeyInfo
omKeyInfo,
public static boolean isImmediateChild(long parentId, long ancestorId) {
return parentId == ancestorId;
}
+
+
+ /**
+ * Check for directory exists with same name, if it exists throw error.
+ *
+ * @param keyName key name
+ * @param ozoneManager Ozone Manager
+ * @param reachedLastPathComponent true if the path component is a fileName
+ * @throws IOException if directory exists with same name
+ */
+ private static void checkDirectoryAlreadyExists(String keyName,
+ OzoneManager ozoneManager, boolean reachedLastPathComponent)
+ throws IOException {
+ // Reached last component, which would be a file. Returns its parentID.
Review comment:
Minor: Comment says Returns its parentID.
In code we just check if lastComponent is true and fileSystem enabled, we
return exception.
And also as current V1 always assumes fs enabled true, do we need &&
ozoneManager.getEnableFileSystemPaths() check?
And also these 3 lines can be in the calling method, as it is not a utility
method which is called by different classes.
##########
File path:
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseV1.java
##########
@@ -0,0 +1,84 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.ozone.om.helpers.OmBucketInfo;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.OmVolumeArgs;
+import org.apache.hadoop.ozone.om.ratis.utils.OzoneManagerRatisUtils;
+import org.apache.hadoop.ozone.om.request.TestOMRequestUtils;
+import
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMResponse;
+import org.apache.hadoop.util.Time;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Tests OMAllocateBlockResponse layout version V1.
+ */
+public class TestOMAllocateBlockResponseV1
+ extends TestOMAllocateBlockResponse {
+
+ // logical ID, which really doesn't exist in dirTable
+ private long parentID = 10;
+ private String fileName = "file1";
+
+ protected OmKeyInfo createOmKeyInfo() throws Exception {
+ // need to initialize parentID
+ String parentDir = keyName;
+ keyName = parentDir + OzoneConsts.OM_KEY_PREFIX + fileName;
+
+ long txnId = 50;
+ long objectId = parentID + 1;
+
+ OmKeyInfo omKeyInfoV1 =
+ TestOMRequestUtils.createOmKeyInfo(volumeName, bucketName, keyName,
+ HddsProtos.ReplicationType.RATIS,
+ HddsProtos.ReplicationFactor.ONE, objectId, parentID,
txnId,
+ Time.now());
+ return omKeyInfoV1;
+ }
+
+ protected String getOpenKey() throws Exception {
+ return omMetadataManager.getOpenFileName(
+ parentID, fileName, clientID);
+ }
+
+ @NotNull
+ protected OMAllocateBlockResponse getOmAllocateBlockResponse(
+ OmKeyInfo omKeyInfo, OmVolumeArgs omVolumeArgs,
+ OmBucketInfo omBucketInfo, OMResponse omResponse) {
+ return new OMAllocateBlockResponseV1(omResponse, omKeyInfo, clientID,
+ omVolumeArgs, omBucketInfo);
+ }
+
+ @NotNull
+ @Override
+ protected OzoneConfiguration getOzoneConfiguration() {
+ OzoneConfiguration config = super.getOzoneConfiguration();
+ config.set(OMConfigKeys.OZONE_OM_LAYOUT_VERSION, "V1");
+ // OzoneManager#start() is not invoked in this test cases. Hence need to
Review comment:
Is this for getKeyTable where we used this in metadatamanager?
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMAllocateBlockRequest.java
##########
@@ -216,17 +216,15 @@ public OMClientResponse
validateAndUpdateCache(OzoneManager ozoneManager,
openKeyInfo.setUpdateID(trxnLogIndex, ozoneManager.isRatisEnabled());
// Add to cache.
- omMetadataManager.getOpenKeyTable().addCacheEntry(
- new CacheKey<>(openKeyName),
- new CacheValue<>(Optional.of(openKeyInfo), trxnLogIndex));
-
+ addOpenTableCacheEntry(trxnLogIndex, omMetadataManager, openKeyName,
+ openKeyInfo);
omBucketInfo.incrUsedBytes(preAllocatedSpace);
omResponse.setAllocateBlockResponse(AllocateBlockResponse.newBuilder()
.setKeyLocation(blockLocation).build());
- omClientResponse = new OMAllocateBlockResponse(omResponse.build(),
- openKeyInfo, clientID, omVolumeArgs, omBucketInfo.copyObject());
+ omClientResponse = getOmClientResponse(clientID, omResponse,
Review comment:
We don't update omVolumeArgs, but still reading and passing to Response,
which is not required.
Not related to your patch BTW, we can fix this in master and get to branch.
##########
File path:
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseV1.java
##########
@@ -0,0 +1,84 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.ozone.om.helpers.OmBucketInfo;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.OmVolumeArgs;
+import org.apache.hadoop.ozone.om.ratis.utils.OzoneManagerRatisUtils;
+import org.apache.hadoop.ozone.om.request.TestOMRequestUtils;
+import
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMResponse;
+import org.apache.hadoop.util.Time;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Tests OMAllocateBlockResponse layout version V1.
+ */
+public class TestOMAllocateBlockResponseV1
+ extends TestOMAllocateBlockResponse {
+
+ // logical ID, which really doesn't exist in dirTable
+ private long parentID = 10;
+ private String fileName = "file1";
+
+ protected OmKeyInfo createOmKeyInfo() throws Exception {
+ // need to initialize parentID
+ String parentDir = keyName;
+ keyName = parentDir + OzoneConsts.OM_KEY_PREFIX + fileName;
+
+ long txnId = 50;
+ long objectId = parentID + 1;
+
+ OmKeyInfo omKeyInfoV1 =
+ TestOMRequestUtils.createOmKeyInfo(volumeName, bucketName, keyName,
+ HddsProtos.ReplicationType.RATIS,
+ HddsProtos.ReplicationFactor.ONE, objectId, parentID,
txnId,
+ Time.now());
+ return omKeyInfoV1;
+ }
+
+ protected String getOpenKey() throws Exception {
+ return omMetadataManager.getOpenFileName(
+ parentID, fileName, clientID);
+ }
+
+ @NotNull
+ protected OMAllocateBlockResponse getOmAllocateBlockResponse(
+ OmKeyInfo omKeyInfo, OmVolumeArgs omVolumeArgs,
+ OmBucketInfo omBucketInfo, OMResponse omResponse) {
+ return new OMAllocateBlockResponseV1(omResponse, omKeyInfo, clientID,
+ omVolumeArgs, omBucketInfo);
+ }
+
+ @NotNull
+ @Override
+ protected OzoneConfiguration getOzoneConfiguration() {
+ OzoneConfiguration config = super.getOzoneConfiguration();
+ config.set(OMConfigKeys.OZONE_OM_LAYOUT_VERSION, "V1");
+ // OzoneManager#start() is not invoked in this test cases. Hence need to
Review comment:
Not understood this comment and why we need to do this, and is there in
many tests, can you provide info why we need to do this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
