[
https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307521#comment-16307521
]
Mike Drob commented on HBASE-19369:
-----------------------------------
When hadoop is configured with Erasure Coding (EC - new in 3.0 and maybe 2.9 I
think) then our WALs don't work because EC doesn't support hsync or hflush. To
get around this, we can still request from the NN that the file we create use
replication instead of EC but we can only do so using the builder API, which
doesn't exist in the older versions that we support.
Running on hadoop 2.8, we can call {{create}} and {{createNonRecursive}} and
the files will be normal (replicated) files.
Running on hadoop 3.0, we can call those same methods and the files will be
either replicated or erasure coded depending on the policy configured.
Running on hadoop 3.0, we can use new builder API to make sure that the files
are replicated.
> HBase Should use Builder Pattern to Create Log Files while using WAL on
> Erasure Coding
> --------------------------------------------------------------------------------------
>
> Key: HBASE-19369
> URL: https://issues.apache.org/jira/browse/HBASE-19369
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Alex Leblang
> Assignee: Alex Leblang
> Attachments: HBASE-19369.master.001.patch,
> HBASE-19369.master.002.patch, HBASE-19369.master.003.patch,
> HBASE-19369.master.004.patch, HBASE-19369.v5.patch, HBASE-19369.v6.patch,
> HBASE-19369.v7.patch, HBASE-19369.v8.patch
>
>
> Right now an HBase instance using the WAL won't function properly in an
> Erasure Coded environment. We should change the following line to use the
> hdfs.DistributedFileSystem builder pattern
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
