[
https://issues.apache.org/jira/browse/HDDS-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383243#comment-17383243
]
Bharat Viswanadham edited comment on HDDS-5393 at 7/19/21, 10:24 AM:
---------------------------------------------------------------------
The problem here is the key create does below.
Key1 create - Blocks Version0 B1(0), B2(0), B3(0)
1st time override
Key1 override - Version0 -> B1(0), B2(0), B3(0), Version1 -> B1(0), B2(0),
B3(0), New Blocks for this version (B4(1), B5(1), B6(1))
2nd time override
Key1 override - Version0 B1(0), B2(0), B3(0), Version1 -> B1(0), B2(0), B3(0),
New Blocks for this version (B4(1), B5(1), B6(1)), Version 2 -> B1(0), B2(0),
B3(0), B4(1), B5(1), B6(1), B7(2), B8(2), B9(2)
So, the problem is each time a key is override, each version has its version of
blocks along with all older versions.
So, now when key create it returns all this versions. So, the list can become
huge in size, depending on number of times key is override. In this patch I
will fix to return latest version for create key. (I will open a new Jira to
discuss about having each version including all its previous version)
was (Author: bharatviswa):
The problem here is the key create does below.
Key1 create - Blocks Version1 B1(0), B2(0), B3(0)
1st time override
Key1 override - Version1 -> B1(0), B2(0), B3(0), Version2 -> B1(0), B2(0),
B3(0), New Blocks for this version (B4(1), B5(1), B6(1))
2nd time override
Key1 override - Version1 B1(0), B2(0), B3(0), Version2 -> B1(0), B2(0), B3(0),
New Blocks for this version (B4(1), B5(1), B6(1)), Version 3 -> B1(0), B2(0),
B3(0), B4(1), B5(1), B6(1), B7(2), B8(2), B9(2)
So, the problem is each time a key is override, each version has its version of
blocks along with all older versions.
So, now when key create it returns all this versions. So, the list can become
huge in size, depending on number of times key is override. In this patch I
will fix to return latest version for create key. (I will open a new Jira to
discuss about having each version including all its previous version)
> Return latest version of key location for client on createKey
> -------------------------------------------------------------
>
> Key: HDDS-5393
> URL: https://issues.apache.org/jira/browse/HDDS-5393
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: UENISHI Kota
> Assignee: Bharat Viswanadham
> Priority: Major
> Fix For: 1.2.0
>
>
> HDDS-5243 was a patch for omitting unnecessary key locations for clients on
> reading. But the same warning of large response size observed in our cluster
> for putting data. The patch can also be ported for putting data, as long as
> until object versioning is supported.
> My hypothesis is: The large message was originally, and possibly maybe due to
> this warning and sudden connection close from client side on reading large
> message in Hadoop IPC layer, from Ozone Manager - which causes hopeless 15
> retries from RetryInvocationHandler. The retries create another entry in
> OpenKeyTable but they never moved to KeyTable because the key never gets
> commited.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
