[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the
recover-trash command server side handling.
URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r398700964
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMTrashRecoverResponse.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo;
+import org.apache.hadoop.ozone.om.response.OMClientResponse;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.OMResponse;
+import org.apache.hadoop.hdds.utils.db.BatchOperation;
+
+import java.io.IOException;
+import javax.annotation.Nullable;
+import javax.annotation.Nonnull;
+
+/**
+ * Response for RecoverTrash request.
+ */
+public class OMTrashRecoverResponse extends OMClientResponse {
+ private OmKeyInfo omKeyInfo;
+
+ public OMTrashRecoverResponse(@Nullable OmKeyInfo omKeyInfo,
+ @Nonnull OMResponse omResponse) {
+super(omResponse);
+this.omKeyInfo = omKeyInfo;
+ }
+
+ @Override
+ public void addToDBBatch(OMMetadataManager omMetadataManager,
+ BatchOperation batchOperation) throws IOException {
+
+ /* TODO: HDDS-2425. HDDS-2426. */
+String trashKey = omMetadataManager
+.getOzoneKey(omKeyInfo.getVolumeName(),
+omKeyInfo.getBucketName(), omKeyInfo.getKeyName());
+RepeatedOmKeyInfo repeatedOmKeyInfo = omMetadataManager
+.getDeletedTable().get(trashKey);
+omKeyInfo = OmUtils.prepareKeyForRecover(omKeyInfo, repeatedOmKeyInfo);
+omMetadataManager.getDeletedTable()
+.deleteWithBatch(batchOperation, omKeyInfo.getKeyName());
+/* TODO: trashKey should be updated to destinationBucket. */
Review comment:
I am fine with recovering last delete key if that is the expected behavior.
> (And when recovering the latest key, I think we should clear the old
deleted key.)
We should not delete the other keys, as those keys will be picked by
background trash service and the data for those keys need to be deleted.
And also doing this way, is also not correct from my understanding, let us
say, we put those keys in delete table, and background delete key service will
pick them up and send to SCM for deletion, at this point we got a recover trash
command, so there is a chance that we recover the key which might have no data,
as we submitted the request to SCM for deletion, and SCM, in turn, it will send
to DN. How we shall handle this kind of scenarios?
Because deletion from delete table will happen when key purge request
happens.
Code snippet link [#link](
https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java#L167
)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the
recover-trash command server side handling.
URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r379263659
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMTrashRecoverResponse.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo;
+import org.apache.hadoop.ozone.om.response.OMClientResponse;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.OMResponse;
+import org.apache.hadoop.hdds.utils.db.BatchOperation;
+
+import java.io.IOException;
+import javax.annotation.Nullable;
+import javax.annotation.Nonnull;
+
+/**
+ * Response for RecoverTrash request.
+ */
+public class OMTrashRecoverResponse extends OMClientResponse {
+ private OmKeyInfo omKeyInfo;
+
+ public OMTrashRecoverResponse(@Nullable OmKeyInfo omKeyInfo,
Review comment:
How we get omKeyInfo value, will be done in later jiras?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the
recover-trash command server side handling.
URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r379263262
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMTrashRecoverRequest.java
##
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.request.key;
+
+import java.io.IOException;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.ozone.om.ratis.utils.OzoneManagerDoubleBufferHelper;
+import org.apache.hadoop.ozone.om.response.key.OMTrashRecoverResponse;
+import org.apache.hadoop.ozone.security.acl.IAccessAuthorizer;
+import org.apache.hadoop.ozone.security.acl.OzoneObj;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import org.apache.hadoop.ozone.om.response.OMClientResponse;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.RecoverTrashRequest;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.OMResponse;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.OMRequest;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type;
+import
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Status;
+
+import static
org.apache.hadoop.ozone.om.lock.OzoneManagerLock.Resource.BUCKET_LOCK;
+
+/**
+ * Handles RecoverTrash request.
+ */
+public class OMTrashRecoverRequest extends OMKeyRequest {
+ private static final Logger LOG =
+ LoggerFactory.getLogger(OMTrashRecoverRequest.class);
+
+ public OMTrashRecoverRequest(OMRequest omRequest) {
+super(omRequest);
+ }
+
+ @Override
+ public OMRequest preExecute(OzoneManager ozoneManager) {
+RecoverTrashRequest recoverTrashRequest = getOmRequest()
+.getRecoverTrashRequest();
+Preconditions.checkNotNull(recoverTrashRequest);
+
+return getOmRequest().toBuilder().build();
+ }
+
+ @Override
+ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
+ long transactionLogIndex,
+ OzoneManagerDoubleBufferHelper ozoneManagerDoubleBufferHelper) {
+RecoverTrashRequest recoverTrashRequest = getOmRequest()
+.getRecoverTrashRequest();
+Preconditions.checkNotNull(recoverTrashRequest);
+
+String volumeName = recoverTrashRequest.getVolumeName();
+String bucketName = recoverTrashRequest.getBucketName();
+String keyName = recoverTrashRequest.getKeyName();
+String destinationBucket = recoverTrashRequest.getDestinationBucket();
+
+/** TODO: HDDS-2818. New Metrics for Trash Key Recover and Fails.
+ * OMMetrics omMetrics = ozoneManager.getMetrics();
+ */
+
+OMResponse.Builder omResponse = OMResponse.newBuilder()
+.setCmdType(Type.RecoverTrash).setStatus(Status.OK)
+.setSuccess(true);
+
+OMMetadataManager omMetadataManager = ozoneManager.getMetadataManager();
+boolean acquireLock = false;
+OMClientResponse omClientResponse = null;
+try {
+
+ // check Acl
+ checkKeyAcls(ozoneManager, volumeName, destinationBucket, keyName,
Review comment:
For checkAcl, the key does not exist, checkKeyAcls will fail with
KeyNotFound.
So, I think we should check Bucket write Acls here?
cc @anuengineer
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the
recover-trash command server side handling.
URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r379262796
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMTrashRecoverResponse.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo;
+import org.apache.hadoop.ozone.om.response.OMClientResponse;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.OMResponse;
+import org.apache.hadoop.hdds.utils.db.BatchOperation;
+
+import java.io.IOException;
+import javax.annotation.Nullable;
+import javax.annotation.Nonnull;
+
+/**
+ * Response for RecoverTrash request.
+ */
+public class OMTrashRecoverResponse extends OMClientResponse {
+ private OmKeyInfo omKeyInfo;
+
+ public OMTrashRecoverResponse(@Nullable OmKeyInfo omKeyInfo,
+ @Nonnull OMResponse omResponse) {
+super(omResponse);
+this.omKeyInfo = omKeyInfo;
+ }
+
+ @Override
+ public void addToDBBatch(OMMetadataManager omMetadataManager,
+ BatchOperation batchOperation) throws IOException {
+
+ /* TODO: HDDS-2425. HDDS-2426. */
+String trashKey = omMetadataManager
+.getOzoneKey(omKeyInfo.getVolumeName(),
+omKeyInfo.getBucketName(), omKeyInfo.getKeyName());
+RepeatedOmKeyInfo repeatedOmKeyInfo = omMetadataManager
+.getDeletedTable().get(trashKey);
+omKeyInfo = OmUtils.prepareKeyForRecover(omKeyInfo, repeatedOmKeyInfo);
+omMetadataManager.getDeletedTable()
+.deleteWithBatch(batchOperation, omKeyInfo.getKeyName());
+/* TODO: trashKey should be updated to destinationBucket. */
Review comment:
One question:
1. if the key is created, deleted, the key is created and the key is
deleted. Now when recover, which omKeyInfo will be used from the delete table.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling. URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r379261796 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManager.java ## @@ -169,6 +169,18 @@ OmKeyLocationInfo allocateBlock(OmKeyArgs args, long clientID, List listTrash(String volumeName, String bucketName, String startKeyName, String keyPrefix, int maxKeys) throws IOException; + /** Review comment: Same here, this can be removed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the
recover-trash command server side handling.
URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r379261737
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##
@@ -871,6 +871,19 @@ private boolean isKeyEmpty(OmKeyInfo keyInfo) {
startKeyName, keyPrefix, maxKeys);
}
+ @Override
+ public boolean recoverTrash(String volumeName, String bucketName,
Review comment:
We don't need this any more, after adding OMTrashRecoverRequest.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.
bharatviswa504 commented on a change in pull request #399: HDDS-2424. Add the
recover-trash command server side handling.
URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r379261600
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -2237,13 +2237,22 @@ public void deleteKey(OmKeyArgs args) throws
IOException {
}
}
- // TODO: HDDS-2424. recover-trash command server side handling.
@Override
public boolean recoverTrash(String volumeName, String bucketName,
String keyName, String destinationBucket) throws IOException {
-boolean recoverOperation = true;
-return recoverOperation;
+if (isAclEnabled) {
Review comment:
We don't need this anymore. As OMTrashRecoverRequest is handling recover
request.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
