[
https://issues.apache.org/jira/browse/HDFS-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HDFS-3120:
------------------------------
Description:
The work on branch-20-append was to support *sync*, for durable HBase WALs, not
*append*. The branch-20-append implementation is known to be buggy. There's
been confusion about this, we often answer queries on the list [like
this|http://search-hadoop.com/m/wfed01VOIJ5]. Unfortunately, the way to enable
correct sync on branch-1 for HBase is to set dfs.support.append to true in your
config, which has the side effect of enabling append (which we don't want to
do).
Let's add a new *dfs.support.sync* option that enables working sync (which is
basically the current dfs.support.append flag modulo one place where it's not
referring to sync). For compatibility, if dfs.support.append is set,
dfs.support.sync will be set as well. This way someone can enable sync for
HBase and still keep the current behavior that if dfs.support.append is not set
then an append operation will result in an IOE indicating append is not
supported. We should do this on trunk as well, as there's no reason to conflate
hsync and append with a single config even if append works.
was:
The work on branch-20-append was to support *sync*, for durable HBase WALs, not
*append*. The branch-20-append implementation is known to be buggy. There's
been confusion about this, we often answer queries on the list [like
this|http://search-hadoop.com/m/wfed01VOIJ5]. Unfortunately, the way to enable
correct sync on branch-1 for HBase is to set dfs.support.append to true in your
config, which has the side effect of enabling append (which we don't want to
do).
Let's add a new *dfs.support.hsync* option that enables working sync (which is
basically the current dfs.support.append flag modulo one place where it's not
referring to sync). For compatibility, if dfs.support.append is set,
dfs.support.sync will be set as well. This way someone can enable sync for
HBase and still keep the current behavior that if dfs.support.append is not set
then an append operation will result in an IOE indicating append is not
supported. We should do this on trunk as well, as there's no reason to conflate
hsync and append with a single config even if append works.
Target Version/s: 1.1.0, 2.0.0 (was: 2.0.0, 1.1.0)
For 1.x:
- Add a dfs.support.sync option and enable it by default
For 2.x:
- Make hsync/hflush behavior independent of whether dfs.support.appends is
enabled, ie you can turn append off and hsync/hflush still work
Note this does not add a dfs.support.sync option to 2.x This would be useful to
people who want to disable both append and sync, eg to make a couple paths
cheaper. I think we should not add this option unless it's requested / needed
in the future. Note that this code path is already enabled by default, so the
change is just that people can currently disable all sync/append code paths by
disabling append, now they would just be disabling the append-specific code
paths.
> Provide ability to enable sync without append
> ---------------------------------------------
>
> Key: HDFS-3120
> URL: https://issues.apache.org/jira/browse/HDFS-3120
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 1.0.1
> Reporter: Eli Collins
> Assignee: Eli Collins
>
> The work on branch-20-append was to support *sync*, for durable HBase WALs,
> not *append*. The branch-20-append implementation is known to be buggy.
> There's been confusion about this, we often answer queries on the list [like
> this|http://search-hadoop.com/m/wfed01VOIJ5]. Unfortunately, the way to
> enable correct sync on branch-1 for HBase is to set dfs.support.append to
> true in your config, which has the side effect of enabling append (which we
> don't want to do).
> Let's add a new *dfs.support.sync* option that enables working sync (which is
> basically the current dfs.support.append flag modulo one place where it's not
> referring to sync). For compatibility, if dfs.support.append is set,
> dfs.support.sync will be set as well. This way someone can enable sync for
> HBase and still keep the current behavior that if dfs.support.append is not
> set then an append operation will result in an IOE indicating append is not
> supported. We should do this on trunk as well, as there's no reason to
> conflate hsync and append with a single config even if append works.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
