[
https://issues.apache.org/jira/browse/HDDS-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751574#comment-17751574
]
Ashish Kumar commented on HDDS-5877:
------------------------------------
Hi [~georgeJahad] is this issue still happening?
> listStatus() race condition when OZONE_OM_ENABLE_FILESYSTEM_PATHS disabled
> --------------------------------------------------------------------------
>
> Key: HDDS-5877
> URL: https://issues.apache.org/jira/browse/HDDS-5877
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Filesystem
> Reporter: George Jahad
> Priority: Minor
>
> Under certain circumstances, listStatus() fails to return results until data
> is transferred from the cache to the keyTable.
> *Details*
> When commitKey() is called, a key is first written to the keyTableCache and
> then to the keyTable when the double buffer is flushed.
> If OZONE_OM_ENABLE_FILESYSTEM_PATHS is enabled, and the key has a
> superdirectory, the superdirectory's are added to cache during the openKey()
> call, here:
> [https://github.com/apache/ozone/blob/af5b48e4571e74fcad2e4546ff0efe79c3a96350/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java#L270]
> If OZONE_OM_ENABLE_FILESYSTEM_PATHS is not enabled, the superdirectories are
> not added to the cache. In addition, the cache search method in listStatus()
> is written to skip any keys with an embedded "/" here,
> [https://github.com/apache/ozone/blob/af5b48e4571e74fcad2e4546ff0efe79c3a96350/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L2285]
> So, if OZONE_OM_ENABLE_FILESYSTEM_PATHS is disabled, and a key with an
> embedded "/" is commited, and it is still in the cache, listStatus() won't
> see it or it's superdirectory.
>
> *Other notes*
> Since the test passes once the key is in the table, it can be worked around
> by sleeping/retrying, but probably listStatus() itself should be modified so
> a read from the cache works the same as a read from the keyTable.
> This test exhibits the problem:
> TestOzoneFileSystem::testListStatusWithIntermediateDir:
> [https://github.com/apache/ozone/blob/295492ec87429c9baeb12246c870231d35641535/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java#L588]
> Note that the test incorrectly failed to show the problem until it was
> updated to stop using the pre HA code. (That code doesn't use the cache, so
> the problem was never seen.)
> It is also possible that listStatus() just shouldn't be used when
> OZONE_OM_ENABLE_FILESYSTEM_PATHS is disabled. In that case, maybe it should
> throw an exception.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
