[
https://issues.apache.org/jira/browse/HDFS-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17841557#comment-17841557
]
wangzhihui commented on HDFS-17463:
-----------------------------------
[~ste...@apache.org] I get it. Thanks for your remind.
> Support the switch StringTable Split ID feature
> -----------------------------------------------
>
> Key: HDFS-17463
> URL: https://issues.apache.org/jira/browse/HDFS-17463
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.2.0, 3.3.5, 3.3.3, 3.3.4
> Reporter: wangzhihui
> Priority: Major
> Labels: pull-request-available
> Attachments: Image_struct.png, error.png
>
>
> desc:
> *
> Hadoop 3.2 introduced optimization features for HDFS StringTable
> (b60ca37914b22550e3630fa02742d40697decb3), It resulted in lower versions of
> Hadoop upgraded to 3.2 and later versions not supporting downgrade
> operations.
> !error.png!
> * This issue has also been discussed in HDFS-14831, and it is recommended to
> revert the feature, but it cannot fundamentally solve the problem。
> *
> Therefore, we have added an optimization to support downgrading
>
> Solution:
> * First, we will add the "dfs. image. save. splitId. stringTable" conf
> switch "StringTable optimization feature" is enabled
> * When the conf value is false, an Image file compatible with lower versions
> of HDFS is generated to support downgrading.
> *
> The difference in HDFS Image file format between Hadoop 3.1.1 and Hadoop 3.2
> is shown in the following figure.
> * With the sub-sections feature introduced in HDFS-14617, Protobuf can
> support compatible reading.
> *
> The data structure causing incompatible differences is mainly StringTable.
> !Image_struct.png|width=396,height=163!
> * In "dfs.image.save.splitId.stringTable = false " the Id generation order
> of StringTable starts from 0 to Integer.Max. When true, the Id value range
> follows the latest rules.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
