[
https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164816#comment-14164816
]
Lars Hofhansl commented on HBASE-5954:
--------------------------------------
I'd like to pick this up again.
In HBase trunk (2.x) can I assume we'll only support versions of Hadoop that
have HDFS-744 (i.e. 2.0.2 or later). I think that is the case. If that is
indeed the case I can do away with all reflection and just use the APIs that
allow me to set.
Note that this puts the performance burden on the client. The client force
* sync every edit -> slow
* batch the edits -> caused only one fsync
* perform write with flush only followed by an write with forced sync -> that
will force all prior edit to disk as well (works as we'll automatically sync a
block when closed in this mode)
I need to check whether sync-behind-writes can be enabled on a per file basis
in HDFS. Sync-on-close with sync-behind-writes is extremely slow.
In fact as discussed above sync-on-close and sync-behind-writes should be
enabled by default in all HDFS installations. Since HDFS writes immutable files
only there is no use hanging on dirty blocks (in the hopes that they'll be
updated again before they get written to disk, and thus sync-behind-writes
should not slow things down - it's possible that there're other effects of
course).
> Allow proper fsync support for HBase
> ------------------------------------
>
> Key: HBASE-5954
> URL: https://issues.apache.org/jira/browse/HBASE-5954
> Project: HBase
> Issue Type: Improvement
> Components: HFile, wal
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 5954-trunk-hdfs-trunk-v2.txt,
> 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt,
> 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt,
> 5954-trunk-hdfs-trunk.txt, hbase-hdfs-744.txt
>
>
> At least get recommendation into 0.96 doc and some numbers running w/ this
> hdfs feature enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
