[ https://issues.apache.org/jira/browse/HDFS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975451#comment-15975451 ]
Andrew Wang commented on HDFS-11643: ------------------------------------ Hi Sammi, took a deeper look at this, mostly just nitty and API stuff: HdfsDataOutputStreamBuilder: * We can simplify and rename the setter {{enforceAsReplicatedFile(Boolean)}} to just be {{replicate()}} with no parameters, since the default is false. I think this could also just be a primitive boolean, since the {{dfs.create}} takes a primitive boolean rather than a nullable Boolean. * asReplicatedFile, recommend we rename to {{shouldReplicate}} and make the getter protected * We also should add some Javadoc to {{replicateFile}} and {{setEcPolicyName}} to explain what they do and that they are exclusive. I'd prefer to be more consistent about {{ecPolicyName}} and {{asReplicated}} being exclusive parameters and throwing an exception. We fixed the builder, but a few other places I noticed: * ClientProtocol: docs say that ecPolicyName is ignored, instead, let's make it clear that it's invalid to set {{asReplicated}} to true and also a non-null {{ecPolicyName}}. * Builder#build: Please add some additional help text for the user when the Precondition check fires * FSNamesystem#startFileInt: Let's check exclusivity here as well, can do it outside the write lock where we do the invalid path check. Tests: * Note that the DiskBalancer is different from the Balancer. We should be adding a test to TestBalancer instead. Recommend reformatting these chained calls in TestErasureCodingPolicies like this for clarity: {noformat} fs.newFSDataOutputStreamBuilder(filePath).setEcPolicyName(ecPolicyName) .enforceAsReplicatedFile(false).build().close(); {noformat} {noformat} fs.newFSDataOutputStreamBuilder(filePath) .setEcPolicyName(ecPolicyName) .enforceAsReplicatedFile(false) .build() .close(); {noformat} > Balancer fencing fails when writing erasure coded lock file > ----------------------------------------------------------- > > Key: HDFS-11643 > URL: https://issues.apache.org/jira/browse/HDFS-11643 > Project: Hadoop HDFS > Issue Type: Bug > Components: balancer & mover, erasure-coding > Affects Versions: 3.0.0-alpha1 > Reporter: Andrew Wang > Assignee: SammiChen > Priority: Blocker > Labels: hdfs-ec-3.0-must-do > Attachments: HDFS-11643.001.patch, HDFS-11643.002.patch, > HDFS-11643.003.patch > > > At startup, the balancer writes its hostname to the lock file and calls > hflush(). hflush is not supported for EC files, so this fails when the entire > filesystem is erasure coded. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org