[jira] [Commented] (HDFS-8259) Erasure Coding: System Test of reading EC file
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639798#comment-14639798 ] GAO Rui commented on HDFS-8259: --- Hi [~xinwei], this jira share the same idea of HDFS-8260 . Erasure Coding: System Test of reading EC file -- Key: HDFS-8259 URL: https://issues.apache.org/jira/browse/HDFS-8259 Project: Hadoop HDFS Issue Type: Test Affects Versions: HDFS-7285 Reporter: GAO Rui Assignee: Xinwei Qin Attachments: HDFS-8259-HDFS-7285.001.patch 1. Normally reading EC file(reading without datanote failure and no need of recovery) 2. Reading EC file with datanode failure. 3. Reading EC file with data block recovery by decoding from parity blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8259) Erasure Coding: System Test of reading EC file
[
https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618434#comment-14618434
]
Hadoop QA commented on HDFS-8259:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 5m 47s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:red}-1{color} | release audit | 0m 13s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 31s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 1m 20s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 68m 33s | Tests failed in hadoop-hdfs. |
| | | 89m 58s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestFileAppend4 |
| | hadoop.hdfs.TestRead |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshot |
| | hadoop.hdfs.server.namenode.TestSecondaryWebUi |
| | hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional |
| | hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| | hadoop.hdfs.server.datanode.TestRefreshNamenodes |
| | hadoop.hdfs.TestHdfsAdmin |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| | hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler |
| | hadoop.hdfs.TestClientReportBadBlock |
| | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| | hadoop.hdfs.server.namenode.TestFSEditLogLoader |
| | hadoop.hdfs.server.namenode.TestAuditLogger |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy |
| | hadoop.hdfs.server.blockmanagement.TestDatanodeManager |
| | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles |
| | hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup |
| |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement |
| | hadoop.hdfs.server.namenode.TestNameNodeRpcServer |
| | hadoop.hdfs.TestSafeModeWithStripedFile |
| | hadoop.hdfs.TestFileAppendRestart |
| | hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot |
| | hadoop.hdfs.server.namenode.ha.TestHAFsck |
| | hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | hadoop.hdfs.protocol.TestBlockListAsLongs |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| | hadoop.hdfs.server.namenode.TestHDFSConcat |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.TestRefreshCallQueue |
| | hadoop.hdfs.TestListFilesInDFS |
| | hadoop.hdfs.server.namenode.TestAddBlock |
| | hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.TestNameEditsConfigs |
| | hadoop.hdfs.TestMiniDFSCluster |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.security.TestPermissionSymlinks |
| | hadoop.hdfs.TestDFSRollback |
| | hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots |
| | hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica |
| | hadoop.hdfs.TestFileConcurrentReader |
| | hadoop.hdfs.TestFileAppend2 |
| | hadoop.hdfs.server.datanode.TestDataNodeExit |
| | hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA |
| | hadoop.hdfs.TestGetFileChecksum |
| | hadoop.hdfs.server.namenode.TestSnapshotPathINodes |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.security.TestRefreshUserMappings |
| |
[jira] [Commented] (HDFS-8259) Erasure Coding: System Test of reading EC file
[
https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618181#comment-14618181
]
Xinwei Qin commented on HDFS-8259:
---
In this patch, the test methods {{testReadCorruptedData*()}} will fail to read
a EC file with corrupted blocks, I have created HDFS-8732 to address it.
Erasure Coding: System Test of reading EC file
--
Key: HDFS-8259
URL: https://issues.apache.org/jira/browse/HDFS-8259
Project: Hadoop HDFS
Issue Type: Test
Affects Versions: HDFS-7285
Reporter: GAO Rui
Assignee: Xinwei Qin
Attachments: HDFS-8259-HDFS-7285.001.patch
1. Normally reading EC file(reading without datanote failure and no need of
recovery)
2. Reading EC file with datanode failure.
3. Reading EC file with data block recovery by decoding from parity blocks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8259) Erasure Coding: System Test of reading EC file
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618082#comment-14618082 ] Xinwei Qin commented on HDFS-8259: --- Attached the patch to review. About reading a file with some datanodes failure or some blocks corrupted or some blocks deleted. Erasure Coding: System Test of reading EC file -- Key: HDFS-8259 URL: https://issues.apache.org/jira/browse/HDFS-8259 Project: Hadoop HDFS Issue Type: Test Affects Versions: HDFS-7285 Reporter: GAO Rui Assignee: Xinwei Qin Attachments: HDFS-8259-HDFS-7285.001.patch 1. Normally reading EC file(reading without datanote failure and no need of recovery) 2. Reading EC file with datanode failure. 3. Reading EC file with data block recovery by decoding from parity blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
