[
https://issues.apache.org/jira/browse/HBASE-15029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyin Xu updated HBASE-15029:
-------------------------------
Description:
In HBase, a list of default configuration values are inconsistent with what is
described in the docs ({{hbase-default.xml}}) and the online docs
(http://hbase.apache.org/book.html#config.files).
This is really confusing to users like me, as we usually only rely on the docs.
*1. {{hbase.lease.recovery.dfs.timeout}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{64000}},
while it is {{61000}} in the code,
{code:title=FSHDFSUtils.java (the only usage)|borderStyle=solid}
// This should be set to how long it'll take for us to timeout against
primary datanode if it
// is dead. We set it to 61 seconds, 1 second than the default
READ_TIMEOUT in HDFS, the
// default value for DFS_CLIENT_SOCKET_TIMEOUT_KEY. If recovery is still
failing after this
// timeout, then further recovery will take liner backoff with this base,
to avoid endless
// preemptions when this value is not properly configured.
long subsequentPauseBase = conf.getLong("hbase.lease.recovery.dfs.timeout",
61 * 1000);
{code}
The comments confirm that the value should be {{61000}} rather than {{64000}}.
\\
\\
*2. {{hbase.client.retries.number}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
while it is {{31}} in the code,
{code:title=HConstants.java|borderStyle=solid}
/**
* Default value of {@link #HBASE_CLIENT_RETRIES_NUMBER}.
*/
public static final int DEFAULT_HBASE_CLIENT_RETRIES_NUMBER = 31;
{code}
All the places using {{hbase.client.retries.number}} take
{{DEFAULT_HBASE_CLIENT_RETRIES_NUMBER}} as the default value.
\\
\\
*3. {{zookeeper.session.timeout}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{90000}},
while it is {{180000}} in the code,
{code:title=HConstants.java|borderStyle=solid}
/** Configuration key for ZooKeeper session timeout */
public static final String ZK_SESSION_TIMEOUT = "zookeeper.session.timeout";
/** Default value for ZooKeeper session timeout */
public static final int DEFAULT_ZK_SESSION_TIMEOUT = 180 * 1000;
{code}
All the places using {{zookeeper.session.timeout}} take
{{DEFAULT_ZK_SESSION_TIMEOUT}}.
\\
\\
*4. {{hbase.hstore.blockingStoreFiles}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
while it is {{7}} in the code,
{code:title=HStore.java|borderStyle=solid}
public static final String BLOCKING_STOREFILES_KEY =
"hbase.hstore.blockingStoreFiles";
public static final int DEFAULT_BLOCKING_STOREFILE_COUNT = 7;
{code}
Same as #3 and #4.
\\
\\
*5. {{hbase.http.max.threads}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
while {{10}} is never used in the code.
The code logic is a little bit complex:
{code:title=HttpServer.java (the only usage)|borderStyle=solid}
static final String HTTP_MAX_THREADS = "hbase.http.max.threads";
...
int maxThreads = conf.getInt(HTTP_MAX_THREADS, -1);
// If HTTP_MAX_THREADS is not configured, QueueThreadPool() will use the
// default value (currently 250).
QueuedThreadPool threadPool = maxThreads == -1 ? new QueuedThreadPool()
: new QueuedThreadPool(maxThreads);
{code}
Basically, if the user does not set the value, {{maxThreads}} would be {{-1}}
which is equavalent to the default value of {{_maxThreads}} in the
{{QueuedThreadPool}} class (it is {{250}} right now).
\\
\\
*6. {{hbase.ipc.server.callqueue.handler.factor}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{0.1}},
while it is {{0}} in the code,
{code:title=SimpleRpcScheduler.java|borderStyle=solid}
public static final String CALL_QUEUE_HANDLER_FACTOR_CONF_KEY =
"hbase.ipc.server.callqueue.handler.factor";
...
float callQueuesHandlersFactor =
conf.getFloat(CALL_QUEUE_HANDLER_FACTOR_CONF_KEY, 0);
int numCallQueues = Math.max(1, (int)Math.round(handlerCount *
callQueuesHandlersFactor));
{code}
\\
*7. {{hbase.regionserver.logroll.errors.tolerated}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files],, the value is {{2}},
while it is {{0}} in the code (i.e., the default is not tolerating any
error...),
{code:title=FSHLog.java (the only usage)|borderStyle=solid}
this.closeErrorsTolerated =
conf.getInt("hbase.regionserver.logroll.errors.tolerated", 0);
{code}
This is really confusing as we suppose to have 2 errors tolerated...
\\
\\
*8. {{hbase.snapshot.enabled}}*
I'm not really confident for this one. In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files],, the value is {{true}},
while it is {{false}} in the code
{code:title=SnapshotManager.java (the only usage)|borderStyle=solid}
public static final String HBASE_SNAPSHOT_ENABLED = "hbase.snapshot.enabled";
...
boolean snapshotEnabled = conf.getBoolean(HBASE_SNAPSHOT_ENABLED, false);
{code}
\\
*9. {{hbase.online.schema.update.enable}}*
Again, in both hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files],, the value is {{true}},
while it is {{false}} in both {{TableEventHandler}} and
{{MasterDDLOperationHelper}}:
{code:title=TableEventHandler.java|borderStyle=solid}
private boolean isOnlineSchemaChangeAllowed() {
return this.server.getConfiguration().getBoolean(
"hbase.online.schema.update.enable", false);
}
{code}
{code:title=MasterDDLOperationHelper.java|borderStyle=solid}
public static boolean isOnlineSchemaChangeAllowed(final MasterProcedureEnv
env) {
return env.getMasterServices().getConfiguration()
.getBoolean("hbase.online.schema.update.enable", false);
}
{code}
Thanks!
was:
In HBase, a list of default configuration values are inconsistent with what is
described in the docs ({{hbase-default.xml}}) and the online docs
(http://hbase.apache.org/book.html#config.files).
This is really confusing to users like me, as we usually only rely on the docs.
*1. {{hbase.lease.recovery.dfs.timeout}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{64000}},
while it is {{61000}} in the code,
{code:title=FSHDFSUtils.java (the only usage)|borderStyle=solid}
// This should be set to how long it'll take for us to timeout against
primary datanode if it
// is dead. We set it to 61 seconds, 1 second than the default
READ_TIMEOUT in HDFS, the
// default value for DFS_CLIENT_SOCKET_TIMEOUT_KEY. If recovery is still
failing after this
// timeout, then further recovery will take liner backoff with this base,
to avoid endless
// preemptions when this value is not properly configured.
long subsequentPauseBase = conf.getLong("hbase.lease.recovery.dfs.timeout",
61 * 1000);
{code}
The comments confirm that the value should be {{61000}} rather than {{64000}}.
\\
\\
*2. {{hbase.client.retries.number}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
while it is {{31}} in the code,
{code:title=HConstants.java|borderStyle=solid}
/**
* Default value of {@link #HBASE_CLIENT_RETRIES_NUMBER}.
*/
public static final int DEFAULT_HBASE_CLIENT_RETRIES_NUMBER = 31;
{code}
All the places using {{hbase.client.retries.number}} take
{{DEFAULT_HBASE_CLIENT_RETRIES_NUMBER}} as the default value.
\\
\\
*3. {{zookeeper.session.timeout}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{90000}},
while it is {{180000}} in the code,
{code:title=HConstants.java|borderStyle=solid}
/** Configuration key for ZooKeeper session timeout */
public static final String ZK_SESSION_TIMEOUT = "zookeeper.session.timeout";
/** Default value for ZooKeeper session timeout */
public static final int DEFAULT_ZK_SESSION_TIMEOUT = 180 * 1000;
{code}
All the places using {{zookeeper.session.timeout}} take
{{DEFAULT_ZK_SESSION_TIMEOUT}}.
\\
\\
*4. {{hbase.hstore.blockingStoreFiles}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
while it is {{7}} in the code,
{code:title=HStore.java|borderStyle=solid}
public static final String BLOCKING_STOREFILES_KEY =
"hbase.hstore.blockingStoreFiles";
public static final int DEFAULT_BLOCKING_STOREFILE_COUNT = 7;
{code}
Same as #3 and #4.
\\
\\
*5. {{hbase.http.max.threads}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
while {{10}} is never used in the code.
The code logic is a little bit complex:
{code:title=HttpServer.java (the only usage)|borderStyle=solid}
static final String HTTP_MAX_THREADS = "hbase.http.max.threads";
...
int maxThreads = conf.getInt(HTTP_MAX_THREADS, -1);
// If HTTP_MAX_THREADS is not configured, QueueThreadPool() will use the
// default value (currently 250).
QueuedThreadPool threadPool = maxThreads == -1 ? new QueuedThreadPool()
: new QueuedThreadPool(maxThreads);
{code}
Basically, if the user does not set the value, {{maxThreads}} would be {{-1}}
which is equavalent to the default value of {{_maxThreads}} in the
{{QueuedThreadPool}} class (it is {{250}} right now).
\\
\\
*6. {{hbase.ipc.server.callqueue.handler.factor}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files], the value is {{0.1}},
while it is {{0}} in the code,
{code:title=.../mapred/SimpleRpcScheduler.java|borderStyle=solid}
public static final String CALL_QUEUE_HANDLER_FACTOR_CONF_KEY =
"hbase.ipc.server.callqueue.handler.factor";
...
float callQueuesHandlersFactor =
conf.getFloat(CALL_QUEUE_HANDLER_FACTOR_CONF_KEY, 0);
int numCallQueues = Math.max(1, (int)Math.round(handlerCount *
callQueuesHandlersFactor));
{code}
\\
*7. {{hbase.regionserver.logroll.errors.tolerated}}*
In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files],, the value is {{2}},
while it is {{0}} in the code (i.e., the default is not tolerating any
error...),
{code:title=.java (the only usage)|borderStyle=solid}
this.closeErrorsTolerated =
conf.getInt("hbase.regionserver.logroll.errors.tolerated", 0);
{code}
This is really confusing as we suppose to have 2 errors tolerated...
\\
\\
*8. {{hbase.snapshot.enabled}}*
I'm not really confident for this one. In hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files],, the value is {{true}},
while it is {{false}} in the code
{code:title=SnapshotManager.java (the only usage)|borderStyle=solid}
public static final String HBASE_SNAPSHOT_ENABLED = "hbase.snapshot.enabled";
...
boolean snapshotEnabled = conf.getBoolean(HBASE_SNAPSHOT_ENABLED, false);
{code}
*9. {{hbase.online.schema.update.enable}}*
Again, in both hbase-default.xml and [online
docs|http://hbase.apache.org/book.html#config.files],, the value is {{true}},
while it is {{false}} in both {{TableEventHandler}} and
{{MasterDDLOperationHelper}}:
{code:title=TableEventHandler.java|borderStyle=solid}
private boolean isOnlineSchemaChangeAllowed() {
return this.server.getConfiguration().getBoolean(
"hbase.online.schema.update.enable", false);
}
{code}
{code:title=MasterDDLOperationHelper.java|borderStyle=solid}
public static boolean isOnlineSchemaChangeAllowed(final MasterProcedureEnv
env) {
return env.getMasterServices().getConfiguration()
.getBoolean("hbase.online.schema.update.enable", false);
}
{code}
Thanks!
> A number of inconsistent defaults of configuration parameters
> -------------------------------------------------------------
>
> Key: HBASE-15029
> URL: https://issues.apache.org/jira/browse/HBASE-15029
> Project: HBase
> Issue Type: Bug
> Components: defaults, documentation, hbase
> Affects Versions: 1.1.2
> Reporter: Tianyin Xu
>
> In HBase, a list of default configuration values are inconsistent with what
> is described in the docs ({{hbase-default.xml}}) and the online docs
> (http://hbase.apache.org/book.html#config.files).
> This is really confusing to users like me, as we usually only rely on the
> docs.
> *1. {{hbase.lease.recovery.dfs.timeout}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files], the value is {{64000}},
> while it is {{61000}} in the code,
> {code:title=FSHDFSUtils.java (the only usage)|borderStyle=solid}
> // This should be set to how long it'll take for us to timeout against
> primary datanode if it
> // is dead. We set it to 61 seconds, 1 second than the default
> READ_TIMEOUT in HDFS, the
> // default value for DFS_CLIENT_SOCKET_TIMEOUT_KEY. If recovery is still
> failing after this
> // timeout, then further recovery will take liner backoff with this base,
> to avoid endless
> // preemptions when this value is not properly configured.
> long subsequentPauseBase =
> conf.getLong("hbase.lease.recovery.dfs.timeout", 61 * 1000);
> {code}
> The comments confirm that the value should be {{61000}} rather than
> {{64000}}.
> \\
> \\
> *2. {{hbase.client.retries.number}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
> while it is {{31}} in the code,
> {code:title=HConstants.java|borderStyle=solid}
> /**
> * Default value of {@link #HBASE_CLIENT_RETRIES_NUMBER}.
> */
> public static final int DEFAULT_HBASE_CLIENT_RETRIES_NUMBER = 31;
> {code}
> All the places using {{hbase.client.retries.number}} take
> {{DEFAULT_HBASE_CLIENT_RETRIES_NUMBER}} as the default value.
> \\
> \\
> *3. {{zookeeper.session.timeout}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files], the value is {{90000}},
> while it is {{180000}} in the code,
> {code:title=HConstants.java|borderStyle=solid}
> /** Configuration key for ZooKeeper session timeout */
> public static final String ZK_SESSION_TIMEOUT = "zookeeper.session.timeout";
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 180 * 1000;
> {code}
> All the places using {{zookeeper.session.timeout}} take
> {{DEFAULT_ZK_SESSION_TIMEOUT}}.
> \\
> \\
> *4. {{hbase.hstore.blockingStoreFiles}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
> while it is {{7}} in the code,
> {code:title=HStore.java|borderStyle=solid}
> public static final String BLOCKING_STOREFILES_KEY =
> "hbase.hstore.blockingStoreFiles";
> public static final int DEFAULT_BLOCKING_STOREFILE_COUNT = 7;
> {code}
> Same as #3 and #4.
> \\
> \\
> *5. {{hbase.http.max.threads}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files], the value is {{10}},
> while {{10}} is never used in the code.
> The code logic is a little bit complex:
> {code:title=HttpServer.java (the only usage)|borderStyle=solid}
> static final String HTTP_MAX_THREADS = "hbase.http.max.threads";
> ...
> int maxThreads = conf.getInt(HTTP_MAX_THREADS, -1);
> // If HTTP_MAX_THREADS is not configured, QueueThreadPool() will use the
> // default value (currently 250).
> QueuedThreadPool threadPool = maxThreads == -1 ? new QueuedThreadPool()
> : new QueuedThreadPool(maxThreads);
> {code}
> Basically, if the user does not set the value, {{maxThreads}} would be {{-1}}
> which is equavalent to the default value of {{_maxThreads}} in the
> {{QueuedThreadPool}} class (it is {{250}} right now).
> \\
> \\
> *6. {{hbase.ipc.server.callqueue.handler.factor}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files], the value is {{0.1}},
> while it is {{0}} in the code,
> {code:title=SimpleRpcScheduler.java|borderStyle=solid}
> public static final String CALL_QUEUE_HANDLER_FACTOR_CONF_KEY =
> "hbase.ipc.server.callqueue.handler.factor";
> ...
> float callQueuesHandlersFactor =
> conf.getFloat(CALL_QUEUE_HANDLER_FACTOR_CONF_KEY, 0);
> int numCallQueues = Math.max(1, (int)Math.round(handlerCount *
> callQueuesHandlersFactor));
> {code}
> \\
> *7. {{hbase.regionserver.logroll.errors.tolerated}}*
> In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files],, the value is {{2}},
> while it is {{0}} in the code (i.e., the default is not tolerating any
> error...),
> {code:title=FSHLog.java (the only usage)|borderStyle=solid}
> this.closeErrorsTolerated =
> conf.getInt("hbase.regionserver.logroll.errors.tolerated", 0);
> {code}
> This is really confusing as we suppose to have 2 errors tolerated...
> \\
> \\
> *8. {{hbase.snapshot.enabled}}*
> I'm not really confident for this one. In hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files],, the value is {{true}},
> while it is {{false}} in the code
> {code:title=SnapshotManager.java (the only usage)|borderStyle=solid}
> public static final String HBASE_SNAPSHOT_ENABLED =
> "hbase.snapshot.enabled";
> ...
> boolean snapshotEnabled = conf.getBoolean(HBASE_SNAPSHOT_ENABLED, false);
> {code}
> \\
> *9. {{hbase.online.schema.update.enable}}*
> Again, in both hbase-default.xml and [online
> docs|http://hbase.apache.org/book.html#config.files],, the value is {{true}},
> while it is {{false}} in both {{TableEventHandler}} and
> {{MasterDDLOperationHelper}}:
> {code:title=TableEventHandler.java|borderStyle=solid}
> private boolean isOnlineSchemaChangeAllowed() {
> return this.server.getConfiguration().getBoolean(
> "hbase.online.schema.update.enable", false);
> }
> {code}
> {code:title=MasterDDLOperationHelper.java|borderStyle=solid}
> public static boolean isOnlineSchemaChangeAllowed(final MasterProcedureEnv
> env) {
> return env.getMasterServices().getConfiguration()
> .getBoolean("hbase.online.schema.update.enable", false);
> }
> {code}
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
