[
https://issues.apache.org/jira/browse/HDFS-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HDFS-1562:
------------------------------
Attachment: hdfs-1562-4.patch
Thanks Matt.
1. DFSTestUtil#readFile will reads as much data as the file has, 1024 is the
buffer size used by copyBytes not the amount to read.
2. Good point, renamed.
3. Good suggestions. Incorporated them all.
4. The current timeout is sufficient for this test as I lowered the necessary
intervals in the cluster config (eg to heartbeat more quickly), but now that
these are generic test functions I think you're right, they should have more
slack. I'll double to 20s.
6. Changed the message to "All replicas not corrupted".
7. Good point. I modifed this API to just return needed replications for the
given block. Originally these were separated checks that got coallesced into
one API via the NameNodeAdapter which is why it wasn't per-block, but checking
per-block is better anyway.
Patch attached.
> Add rack policy tests
> ---------------------
>
> Key: HDFS-1562
> URL: https://issues.apache.org/jira/browse/HDFS-1562
> Project: Hadoop HDFS
> Issue Type: Test
> Components: name-node, test
> Affects Versions: 0.23.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-1562-1.patch, hdfs-1562-2.patch, hdfs-1562-3.patch,
> hdfs-1562-4.patch
>
>
> The existing replication tests (TestBlocksWithNotEnoughRacks,
> TestPendingReplication, TestOverReplicatedBlocks, TestReplicationPolicy,
> TestUnderReplicatedBlocks, and TestReplication) are missing tests for rack
> policy violations. This jira adds the following tests which I created when
> generating a new patch for HDFS-15.
> * Test that blocks that have a sufficient number of total replicas, but are
> not replicated cross rack, get replicated cross rack when a rack becomes
> available.
> * Test that new blocks for an underreplicated file will get replicated cross
> rack.
> * Mark a block as corrupt, test that when it is re-replicated that it is
> still replicated across racks.
> * Reduce the replication factor of a file, making sure that the only block
> that is across racks is not removed when deleting replicas.
> * Test that when a block is replicated because a replica is lost due to host
> failure the the rack policy is preserved.
> * Test that when the execss replicas of a block are reduced due to a node
> re-joining the cluster the rack policy is not violated.
> * Test that rack policy is still respected when blocks are replicated due to
> node decommissioning.
> * Test that rack policy is still respected when blocks are replicated due to
> node decommissioning, even when the blocks are over-replicated.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
