zhengchenyu opened a new pull request, #6813: URL: https://github.com/apache/hadoop/pull/6813
### Description of PR I found that if the erasedIndexes distribution is such that the parity index is in front of the data index, ec will produce wrong results when decoding. In fact, [HDFS-15186](https://issues.apache.org/jira/browse/HDFS-15186) has described this problem, but does not fundamentally solve it. The reason is that the code assumes that erasedIndexes is preceded by the data index and followed by parity index. If there is a parity index placed in front of the data index in the incoming code, a calculation error will occur. When we decode the data unit, we multiply the existing data by the decoding matrix. (Look at the formula [doc](https://zhengchenyu.github.io/2024/05/17/ErasuceCode%E7%AE%97%E6%B3%95%E5%AE%9E%E7%8E%B0/) in 1.2) When we decode the parity unit, we multiply the existing data by the decoding matrix, get data unit, then multiply by encoding matrix. (Look at the formula [doc](https://zhengchenyu.github.io/2024/05/17/ErasuceCode%E7%AE%97%E6%B3%95%E5%AE%9E%E7%8E%B0/) in 1.1 and 1.2 ) The calculations for parity and block are different. But They calculate in two separate loops, then the code requires that the data index must precede the parity index. ### How was this patch tested? The TestErasureCodingEncodeAndDecode unit test and the erasure_code_test binary were executed on different machines. The test machines include those with isa-l installed and those without isa-l installed. ### For code changes: - Make erasedIndexes support arbitrary index distribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
