WangYuanben created HDFS-17663:
----------------------------------
Summary: File may be corrupted when using high-data-block EC
policy with ISAL.
Key: HDFS-17663
URL: https://issues.apache.org/jira/browse/HDFS-17663
Project: Hadoop HDFS
Issue Type: Bug
Components: ec, erasure-coding, native
Reporter: WangYuanben
Attachments: 12-4-test.png, 15-1-test.png, files.png
When I define and use the RS-14-2-1024k/RS-13-3-1024k/RS-12-4-1024k EC policy,
I find that files are corrupted when decoded with ISAL. The files can be
retrieved and have the same size as the original, but their MD5 checksums
differ.
To verify that the encoding is correct, I perform a cross-validation: I encode
the file both with and without ISAL, then decode it with and without ISAL. This
results in four files, and only the two files decoded with ISAL have different
MD5 checksums compared to the original. The picture shows the result:
!12-4-test.png|width=550,height=333!
For RS-15-1-1024k, there are issues with both encoding and decoding. Only the
file encoded and decoded without ISAL has the same MD5 checksum as the
original. The picture shows the result:
!15-1-test.png|width=550,height=333!
The test files decoded are as below:
!files.png|width=500,height=333!
Maybe we should do something to solve it ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
