[
https://issues.apache.org/jira/browse/HDDS-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272768#comment-17272768
]
Sadanand Shenoy edited comment on HDDS-4751 at 1/27/21, 11:13 AM:
------------------------------------------------------------------
TrashOzoneFileSystem uses RenameKeyRequest for renames and DeleteKeyRequest for
deletes. In the #preExecute() method of these requests,
RenameKeyRequest stores the toKeyName without the normalised path ( without the
"/")
{code:java}
KeyArgs.Builder newKeyArgs = renameKeyArgs.toBuilder()
.setModificationTime(Time.now());
return getOmRequest().toBuilder()
.setRenameKeyRequest(renameKeyRequest.toBuilder()
.setKeyArgs(newKeyArgs))
.setUserInfo(getUserInfo()).build();
{code}
DeleteKeyRequest checks whether key with "/" exists but it doesn't find
because renameKeyRequest hasn't stored it that way and gives a KEY_NOT_FOUND
and this is why test fails when ozone.om.enabled.filesystem.paths is enabled.
{code:java}
OzoneManagerProtocolProtos.KeyArgs.Builder newKeyArgs =
keyArgs.toBuilder().setModificationTime(Time.now())
.setKeyName(validateAndNormalizeKey(
ozoneManager.getEnableFileSystemPaths(), keyArgs.getKeyName()));
return getOmRequest().toBuilder()
.setDeleteKeyRequest(deleteKeyRequest.toBuilder()
.setKeyArgs(newKeyArgs)).setUserInfo(getUserInfo()).build();
{code}
One fix can be that in RenameKeyRequest the toKey is set with the same name
but it has been addressed in [this
issue|https://github.com/apache/ozone/pull/1256].
However, like in o3fs and ofs if the path is a directory it calls
the(deleteObjects()) DeleteKeysRequest where such there's no such check.
Calling DeleteKeysRequest inside TrashOzoneFileSystem is a possible solution
here.
CC: [~msingh]
was (Author: sadanand_shenoy):
TrashOzoneFileSystem uses RenameKeyRequest for renames and DeleteKeyRequest for
deletes. In the #preExecute() method of these requests,
RenameKeyRequest stores the toKeyName without the normalised path ( without the
"/")
{code:java}
KeyArgs.Builder newKeyArgs = renameKeyArgs.toBuilder()
.setModificationTime(Time.now());
return getOmRequest().toBuilder()
.setRenameKeyRequest(renameKeyRequest.toBuilder()
.setKeyArgs(newKeyArgs))
.setUserInfo(getUserInfo()).build();
{code}
DeleteKeyRequest checks whether key with "/" exists but it doesn't find
because renameKeyRequest hasn't stored it that way and gives a KEY_NOT_FOUND
and this is why test fails when ozone.om.enabled.filesystem.paths is enabled.
{code:java}
OzoneManagerProtocolProtos.KeyArgs.Builder newKeyArgs =
keyArgs.toBuilder().setModificationTime(Time.now())
.setKeyName(validateAndNormalizeKey(
ozoneManager.getEnableFileSystemPaths(), keyArgs.getKeyName()));
return getOmRequest().toBuilder()
.setDeleteKeyRequest(deleteKeyRequest.toBuilder()
.setKeyArgs(newKeyArgs)).setUserInfo(getUserInfo()).build();
{code}
One fix can be that in RenameKeyRequest the toKey is set with the same name
but it has been addressed in [this
issue|https://github.com/apache/ozone/pull/1256].
However, like in o3fs and ofs if the path is a directory it calls
the(deleteObjects()) DeleteKeysRequest where such there's no such check.
Calling DeleteKeysRequest inside TrashOzoneFileSystem is a possible solution
here.
> TestOzoneFileSystem#testTrash failed when enabledFileSystemPaths and
> omRatisDisabled
> ------------------------------------------------------------------------------------
>
> Key: HDDS-4751
> URL: https://issues.apache.org/jira/browse/HDDS-4751
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Rakesh Radhakrishnan
> Assignee: Sadanand Shenoy
> Priority: Major
>
> Presently TestOzoneFileSystem all 4 modes are running always with
> {{enabledFileSystemPaths=false}} and {{omRatisEnabled=false}}, which is
> basically due to the wrong init using {{@BeforeClass}}. I've modified the
> test to have proper init/teardown and noticed this failure
> [Build_Check|https://github.com/apache/ozone/pull/1841/checks?check_run_id=1763626625].
> I will raise separate jira to correct TestOzoneFileSystem {{@BeforeClass}}.
> Hits below exception during the run.
> {code}
> 2021-01-26 21:40:00,016 [pool-25-thread-2] ERROR key.OMKeyDeleteRequest
> (OMKeyDeleteRequest.java:validateAndUpdateCache(188)) - Key delete failed.
> Volume:volume49162, Bucket:bucket42587,
> Key:.Trash/rakeshr/210126213900.2021-01-26 21:40:00,016 [pool-25-thread-2]
> ERROR key.OMKeyDeleteRequest
> (OMKeyDeleteRequest.java:validateAndUpdateCache(188)) - Key delete failed.
> Volume:volume49162, Bucket:bucket42587,
> Key:.Trash/rakeshr/210126213900.KEY_NOT_FOUND
> org.apache.hadoop.ozone.om.exceptions.OMException: Key not found at
> org.apache.hadoop.ozone.om.request.key.OMKeyDeleteRequest.validateAndUpdateCache(OMKeyDeleteRequest.java:132)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:227)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequestDirectlyToOM(OzoneManagerProtocolServerSideTranslatorPB.java:246)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:150)
> at
> org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:122)
> at
> org.apache.hadoop.ozone.om.TrashOzoneFileSystem.submitRequest(TrashOzoneFileSystem.java:113)
> at
> org.apache.hadoop.ozone.om.TrashOzoneFileSystem.access$200(TrashOzoneFileSystem.java:65)
> at
> org.apache.hadoop.ozone.om.TrashOzoneFileSystem$DeleteIterator.processKeyPath(TrashOzoneFileSystem.java:450)
> at
> org.apache.hadoop.ozone.om.TrashOzoneFileSystem$OzoneListingIterator.iterate(TrashOzoneFileSystem.java:350)
> at
> org.apache.hadoop.ozone.om.TrashOzoneFileSystem.delete(TrashOzoneFileSystem.java:163)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone.deleteCheckpoint(TrashPolicyOzone.java:269)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone.access$900(TrashPolicyOzone.java:55)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone$Emptier.lambda$run$0(TrashPolicyOzone.java:177)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
