[
https://issues.apache.org/jira/browse/HDFS-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
WangYuanben updated HDFS-17663:
-------------------------------
Attachment: files.png
> File may be corrupted when using high-data-block EC policy with ISAL.
> ---------------------------------------------------------------------
>
> Key: HDFS-17663
> URL: https://issues.apache.org/jira/browse/HDFS-17663
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ec, erasure-coding, native
> Reporter: WangYuanben
> Priority: Major
> Attachments: 12-4-test.png, 15-1-test.png, files.png
>
>
> When I define and use the RS-14-2-1024k/RS-13-3-1024k/RS-12-4-1024k EC
> policy, I find that files are corrupted when decoded with ISAL. The files can
> be retrieved and have the same size as the original, but their MD5 checksums
> differ.
> To verify that the encoding is correct, I perform a cross-validation: I
> encode the file both with and without ISAL, then decode it with and without
> ISAL. This results in four files, and only the two files decoded with ISAL
> have different MD5 checksums compared to the original. The picture shows the
> result:
> !12-4-test.png|width=550,height=333!
>
> For RS-15-1-1024k, there are issues with both encoding and decoding. Only the
> file encoded and decoded without ISAL has the same MD5 checksum as the
> original. The picture shows the result:
> !15-1-test.png|width=550,height=333!
> The test files decoded are as below:
>
> When the number of data blocks is fewer than 12, this issue does not occur.
> Additionally, the number of parity blocks does not seem to affect the issue.
> RS-12-4-1024k, RS-12-3-1024k, and RS-12-2-1024k all exhibit the problem
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
