[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[
https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437470#comment-13437470
]
Matteo Bertozzi commented on HBASE-6567:
@stack the patch contains also the hbase-env.sh with the two new variables a
comment that try to explain what the mlock is used for, but doesn't seems to be
in trunk. Do you think is not useful to expose that? (I've no strong opinion on
expose it or not, since most of the user can live without knowing it)
{code}
+# Uncomment and adjust to keep all the Region Server pages mapped to be memory
resident
+#HBASE_REGIONSERVER_MLOCK=true
+#HBASE_REGIONSERVER_UID=hbase
{code}
make memory locking configuration of regioservers more flexible
---
Key: HBASE-6567
URL: https://issues.apache.org/jira/browse/HBASE-6567
Project: HBase
Issue Type: Improvement
Components: scripts
Affects Versions: 0.96.0
Reporter: Roman Shaposhnik
Assignee: Matteo Bertozzi
Fix For: 0.96.0
Attachments: HBASE-6567-v0.patch
The current implementation of the memory locking feature of regisoservers has
a downside of not being flexible to configure for permanent use. Sure there
is a --mlock flag but that needs to be explicitly passed on every invocation
and thus require extra steps to be configured for permanent use (IOW, there's
not a single env variable I can set to have a desired effect). The only other
alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a
downside of being pretty cryptic to the novice user and has a killer problem
of not explicitly telling higher level scripts (like init.d or upstart ones)
which user the initial hbase process should be executed as.
I propose a very simple solution (which is essentially making --mlock setting
into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that
can be set in hbase-env.sh and has the following semantics:
* [default] not set: mlocking feature is disabled
* set but empty: mlocking feature is enabled and the target user is hbase
* set and not empty: mlocking feature is enabled and the target user is
the value of the variable
Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437576#comment-13437576 ] stack commented on HBASE-6567: -- @Matteo I made a mistake in application. Should be fixed now. Thanks for noticing it. make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Matteo Bertozzi Fix For: 0.96.0 Attachments: HBASE-6567-v0.patch The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437588#comment-13437588 ] Hudson commented on HBASE-6567: --- Integrated in HBase-TRUNK #3240 (See [https://builds.apache.org/job/HBase-TRUNK/3240/]) HBASE-6567 make memory locking configuration of regioservers more flexible; INCOMPLETE APPLICATION (Revision 1374848) Result = FAILURE stack : Files : * /hbase/trunk/conf/hbase-env.sh make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Matteo Bertozzi Fix For: 0.96.0 Attachments: HBASE-6567-v0.patch The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437609#comment-13437609 ] Hudson commented on HBASE-6567: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #137 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/137/]) HBASE-6567 make memory locking configuration of regioservers more flexible; INCOMPLETE APPLICATION (Revision 1374848) Result = FAILURE stack : Files : * /hbase/trunk/conf/hbase-env.sh make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Matteo Bertozzi Fix For: 0.96.0 Attachments: HBASE-6567-v0.patch The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437430#comment-13437430 ] Hudson commented on HBASE-6567: --- Integrated in HBase-TRUNK #3238 (See [https://builds.apache.org/job/HBase-TRUNK/3238/]) HBASE-6567 make memory locking configuration of regioservers more flexible (Revision 1374662) Result = SUCCESS stack : Files : * /hbase/trunk/bin/hbase-config.sh make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Matteo Bertozzi Fix For: 0.96.0 Attachments: HBASE-6567-v0.patch The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437437#comment-13437437 ] Hudson commented on HBASE-6567: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #135 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/135/]) HBASE-6567 make memory locking configuration of regioservers more flexible (Revision 1374662) Result = FAILURE stack : Files : * /hbase/trunk/bin/hbase-config.sh make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Matteo Bertozzi Fix For: 0.96.0 Attachments: HBASE-6567-v0.patch The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433291#comment-13433291 ] Zhihong Ted Yu commented on HBASE-6567: --- Would HBASE_REGIONSERVER_MLOCK_USER be a better name for this new variable ? make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Roman Shaposhnik Fix For: 0.96.0 The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433304#comment-13433304 ] Roman Shaposhnik commented on HBASE-6567: - It would definitely address the issue that is lurking (mainly the fact that the proposed variable serves too purposes -- enabling the feature to begin with and telling the system what is the target user). That said, I feel that HBASE_REGIONSERVER_MLOCK_USER is also not ideal, since it doesn't really communicate clearly what is happening. In my initial proposal I was trying to minimize the # of variables that we would have to introduce but perhaps we should just bite the bullet and have two: * HBASE_REGIONSERVER_MLOCK (binary: empty/not set -- disabled, non empty -- enabled) * HBASE_REGIONSERVER_UID (string: a target user for the running process regardless of any other feature that is enabled) make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Roman Shaposhnik Fix For: 0.96.0 The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6567) make memory locking configuration of regioservers more flexible
[ https://issues.apache.org/jira/browse/HBASE-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433359#comment-13433359 ] Zhihong Ted Yu commented on HBASE-6567: --- Looks like two variable approach would minimize misunderstanding of this feature. make memory locking configuration of regioservers more flexible --- Key: HBASE-6567 URL: https://issues.apache.org/jira/browse/HBASE-6567 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.96.0 Reporter: Roman Shaposhnik Assignee: Roman Shaposhnik Fix For: 0.96.0 The current implementation of the memory locking feature of regisoservers has a downside of not being flexible to configure for permanent use. Sure there is a --mlock flag but that needs to be explicitly passed on every invocation and thus require extra steps to be configured for permanent use (IOW, there's not a single env variable I can set to have a desired effect). The only other alternative -- the explicit setting of HBASE_REGIONSERVER_OPTS -- has a downside of being pretty cryptic to the novice user and has a killer problem of not explicitly telling higher level scripts (like init.d or upstart ones) which user the initial hbase process should be executed as. I propose a very simple solution (which is essentially making --mlock setting into an env. variable): add a variable called HBASE_REGIONSERVER_MLOCK that can be set in hbase-env.sh and has the following semantics: * [default] not set: mlocking feature is disabled * set but empty: mlocking feature is enabled and the target user is hbase * set and not empty: mlocking feature is enabled and the target user is the value of the variable Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
