[
https://issues.apache.org/jira/browse/HDFS-11117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658998#comment-15658998
]
Kai Zheng commented on HDFS-11117:
----------------------------------
Thanks [~Sammi] for the good work!
1. For the following function, by the way better: {{numBytes}} ->
{{numBytesInStrip}}. Similar to {{getRealTotalBlockNum}}.
{code}
- public static short getRealDataBlockNum(int numBytes) {
- return (short) Math.min(NUM_DATA_BLOCKS,
- (numBytes - 1) / BLOCK_STRIPED_CELL_SIZE + 1);
+ public static short getRealDataBlockNum(int numBytes,
+ ErasureCodingPolicy ecPolicy) {
+ return (short) Math.min(ecPolicy.getNumDataUnits(),
+ (numBytes - 1) / ecPolicy.getCellSize() + 1);
}
{code}
2. In {{TestSortLocatedStripedBlock}} and {{TestBlockTokenWithDFSStriped}}:
Why the variables are static? It's the effort here that tries to avoid static,
right. Please check other tests as well. In such tests, even we use the system
default policy right now, we may also try other policies in future.
Could we change by the way to make the variable naming styles be consistent?
{code}
+ private static final short NUM_DATA_BLOCKS =
+ (short) ecPolicy.getNumDataUnits();
+ private static final short NUM_PARITY_BLOCKS =
+ (short) ecPolicy.getNumParityUnits();
+ static final int BLK_GROUP_WIDTH = NUM_DATA_BLOCKS + NUM_PARITY_BLOCKS;
...
+ private final static ErasureCodingPolicy ecPolicy =
+ ErasureCodingPolicyManager.getSystemDefaultPolicy();
+ private final static int dataBlocks = ecPolicy.getNumDataUnits();
+ private final static int parityBlocks = ecPolicy.getNumParityUnits();
+ private final static int cellSize = ecPolicy.getCellSize();
{code}
> Refactor striped file tests to allow flexibly test erasure coding policy
> ------------------------------------------------------------------------
>
> Key: HDFS-11117
> URL: https://issues.apache.org/jira/browse/HDFS-11117
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: SammiChen
> Assignee: SammiChen
> Attachments: HDFS-11117-v1.patch, HDFS-11117-v2.patch
>
>
> This task is going to refactor current striped file test case structures,
> especially {{StripedFileTestUtil}} file which is used in many striped file
> test cases. All current striped file test cases only support one erasure
> coding policy, that's the default RS-DEFAULT-6-3-64k policy. The goal of the
> refactor is to make the structures more convenient to support other erasure
> coding policies, such as XOR policy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
