[
https://issues.apache.org/jira/browse/HDFS-14039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HDFS-14039:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.2.1
3.3.0
3.1.2
3.0.4
Status: Resolved (was: Patch Available)
Committed to trunk all the way down to branch-3.0.
There were trivial conflicts in branch-3.1, resolved and compiled before
pushing. Also some conflicts in branch-3.0, backported HDFS-13772 first to get
a clean cherry-pick.
Thank you for the great work here, [~knanasi]! Feel free to create a jira to
rethink about consolidating ECPM's internal data structures on trunk. :)
> ec -listPolicies doesn't show correct state for the default policy when the
> default is not RS(6,3)
> --------------------------------------------------------------------------------------------------
>
> Key: HDFS-14039
> URL: https://issues.apache.org/jira/browse/HDFS-14039
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Affects Versions: 3.0.0
> Reporter: Xiao Chen
> Assignee: Kitti Nanasi
> Priority: Major
> Fix For: 3.0.4, 3.1.2, 3.3.0, 3.2.1
>
> Attachments: HDFS-14039.001.patch, HDFS-14039.002.patch,
> HDFS-14039.003.patch, HDFS-14039.004.patch, HDFS-14039.005.patch,
> HDFS-14039.006.patch
>
>
> {noformat}
> $ hdfs ec -listPolicies
> Erasure Coding Policies:
> ErasureCodingPolicy=[Name=RS-10-4-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=10, numParityUnits=4]], CellSize=1048576, Id=5], State=DISABLED
> ErasureCodingPolicy=[Name=RS-3-2-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=3, numParityUnits=2]], CellSize=1048576, Id=2], State=DISABLED
> ErasureCodingPolicy=[Name=RS-6-3-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=6, numParityUnits=3]], CellSize=1048576, Id=1], State=ENABLED
> ErasureCodingPolicy=[Name=RS-LEGACY-6-3-1024k,
> Schema=[ECSchema=[Codec=rs-legacy, numDataUnits=6, numParityUnits=3]],
> CellSize=1048576, Id=3], State=DISABLED
> ErasureCodingPolicy=[Name=XOR-2-1-1024k, Schema=[ECSchema=[Codec=xor,
> numDataUnits=2, numParityUnits=1]], CellSize=1048576, Id=4], State=DISABLED
> $ hdfs ec -enablePolicy -policy XOR-2-1-1024k
> Erasure coding policy XOR-2-1-1024k is enabled
> $ hdfs ec -listPolicies
> Erasure Coding Policies:
> ErasureCodingPolicy=[Name=RS-10-4-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=10, numParityUnits=4]], CellSize=1048576, Id=5], State=DISABLED
> ErasureCodingPolicy=[Name=RS-3-2-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=3, numParityUnits=2]], CellSize=1048576, Id=2], State=DISABLED
> ErasureCodingPolicy=[Name=RS-6-3-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=6, numParityUnits=3]], CellSize=1048576, Id=1], State=ENABLED
> ErasureCodingPolicy=[Name=RS-LEGACY-6-3-1024k,
> Schema=[ECSchema=[Codec=rs-legacy, numDataUnits=6, numParityUnits=3]],
> CellSize=1048576, Id=3], State=DISABLED
> ErasureCodingPolicy=[Name=XOR-2-1-1024k, Schema=[ECSchema=[Codec=xor,
> numDataUnits=2, numParityUnits=1]], CellSize=1048576, Id=4], State=ENABLED
> ----------------------------------
> $ #set default to be RS-3-2 for dfs.namenode.ec.system.default.policy, and
> restart NN
> (this seems to be what's triggering the failure)
> -----------------------------------
> $ hdfs ec -listPolicies
> Erasure Coding Policies:
> ErasureCodingPolicy=[Name=RS-10-4-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=10, numParityUnits=4]], CellSize=1048576, Id=5], State=DISABLED
> ErasureCodingPolicy=[Name=RS-3-2-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=3, numParityUnits=2]], CellSize=1048576, Id=2], State=DISABLED
> ErasureCodingPolicy=[Name=RS-6-3-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=6, numParityUnits=3]], CellSize=1048576, Id=1], State=ENABLED
> ErasureCodingPolicy=[Name=RS-LEGACY-6-3-1024k,
> Schema=[ECSchema=[Codec=rs-legacy, numDataUnits=6, numParityUnits=3]],
> CellSize=1048576, Id=3], State=DISABLED
> ErasureCodingPolicy=[Name=XOR-2-1-1024k, Schema=[ECSchema=[Codec=xor,
> numDataUnits=2, numParityUnits=1]], CellSize=1048576, Id=4], State=ENABLED
> $ hdfs ec -enablePolicy -policy RS-3-2-1024k
> Erasure coding policy RS-3-2-1024k is enabled
> $ hdfs ec -listPolicies
> Erasure Coding Policies:
> ErasureCodingPolicy=[Name=RS-10-4-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=10, numParityUnits=4]], CellSize=1048576, Id=5], State=DISABLED
> ErasureCodingPolicy=[Name=RS-3-2-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=3, numParityUnits=2]], CellSize=1048576, Id=2], State=DISABLED
> ErasureCodingPolicy=[Name=RS-6-3-1024k, Schema=[ECSchema=[Codec=rs,
> numDataUnits=6, numParityUnits=3]], CellSize=1048576, Id=1], State=ENABLED
> ErasureCodingPolicy=[Name=RS-LEGACY-6-3-1024k,
> Schema=[ECSchema=[Codec=rs-legacy, numDataUnits=6, numParityUnits=3]],
> CellSize=1048576, Id=3], State=DISABLED
> ErasureCodingPolicy=[Name=XOR-2-1-1024k, Schema=[ECSchema=[Codec=xor,
> numDataUnits=2, numParityUnits=1]], CellSize=1048576, Id=4], State=ENABLED
> {noformat}
> The last 2 should show RS-3-2 as ENABLED. RS-6-3 DISABLED if it's not enabled
> before.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
