[
https://issues.apache.org/jira/browse/HDDS-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18022796#comment-18022796
]
Donal Magennis commented on HDDS-13708:
---------------------------------------
[~adoroszlai] - Yes, that would be my opinion.
> Missing directory from FSO Bucket listStatus calls while performing renames
> ---------------------------------------------------------------------------
>
> Key: HDDS-13708
> URL: https://issues.apache.org/jira/browse/HDDS-13708
> Project: Apache Ozone
> Issue Type: Bug
> Affects Versions: 1.4.0, 1.4.1
> Reporter: Donal Magennis
> Priority: Critical
> Attachments: ListStatusInconsistencyTest.java
>
>
> We encountered an issue in 1.4.0 and 1.4.1 were we receive inconsistent
> directory listings from the FSO bucket listStatus calls. This results in
> missing data in our application.
> This issue does not exist in master or in 1.3.0.
> We have traced that the issue was introduced in
> [https://github.com/apache/ozone/commit/6cdc3795836c115858130ea540d17a7bdf976723]
> and subsequently resolved in
> [https://github.com/apache/ozone/commit/fa9842682a49863809b3bd4b05e238c8a91df6d1].
> This is reproducible in the attached integration test -
> hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/ListStatusInconsistencyTest.java.
> For reference the test:
> creates 50 dirs in setup in the format:
> /basePath/dir1/dir1/key1
> /basePath/dir2/dir2/key2
> ..Etc
> This ensures every directory is explicitly created with some content
> pre-listing.
> The test then spins up some threads to generate create/renanmes concurrently
> across the directories e.g.
> create - /basePath/dir<1..50>/file<1..1000>._copying
> rename - /basePath/dir1/file1._copying /basePath/dir1/file1
> At the same time the main thread is listing /basePath/.
> We expect to see 50 directories every listing, but eventually do not and the
> test fails reliably.
> When renaming a child key, the parent directory can "disappear" from the
> listing of the grandparent. We do not believe this in line with consistency
> guarantees for ozone.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
