[
https://issues.apache.org/jira/browse/HDDS-9176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17756198#comment-17756198
]
Siyao Meng commented on HDDS-9176:
----------------------------------
[~weichiu] I see.
[~sumitagrawl] I was playing with a test of two clients HSync'ing and
committing the same key while I was working on HDDS-9146:
https://github.com/smengcl/hadoop-ozone/blob/HDDS-9146-testTwoClientsHSyncTheSameKey/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestHSync.java#L199
Result looks like this:
{code}
2023-08-18 16:20:59,476 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 1st (client 1 hsync) read: Client
1 wins
2023-08-18 16:20:59,485 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 2nd (client 2 open) read: Client 1
wins
2023-08-18 16:20:59,500 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 3rd (stream 2 write) read: Client
1 wins
2023-08-18 16:20:59,541 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 4th (stream 2 hsync) read: Client
2 wins
2023-08-18 16:20:59,567 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 5th (stream 2 close) read: Client
2 wins
2023-08-18 16:20:59,581 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 6th (client 2 close) read: Client
2 wins
2023-08-18 16:20:59,605 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 7th (stream 1 close) read: Client
1 wins
2023-08-18 16:20:59,628 [main] WARN ozone.TestHSync
(TestHSync.java:readKey(244)) - --- key data 8th (client 3 read) read: Client 1
wins
{code}
Note the it hasn't tested the behavior when the key has two or more blocks.
We need to first establish the intended behavior. And if we would even allow
this. Then work it out from there.
> two client parallel perform commit with Hsync can cause dataloss
> ----------------------------------------------------------------
>
> Key: HDDS-9176
> URL: https://issues.apache.org/jira/browse/HDDS-9176
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sumit Agrawal
> Priority: Major
>
> 2 client performs commit with hsync feature,
> * client 1 perform commit with Hsync with x block, key will be added in
> keyTable
> * client 2 perform commit with Hsync with y block,
> ** will overwrite previous key,
> ** quota can be issue as previous key of client 1 is used to identify space
> usages delta
> * client 1 perform final commit, will be treated as overwrite and remove
> blocks of client 2 data
> * client 2 perform final commit, this will overwrite and remove blocks of
> client 1 data
> So in above flow, it can cause dataloss and quota will be not be counted
> properly for space usages.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
