[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13161195#comment-13161195 ] Hadoop QA commented on HBASE-4415: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505810/HBASE-4415-8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 71 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRollingRestart org.apache.hadoop.hbase.util.TestRegionSplitter org.apache.hadoop.hbase.io.hfile.TestLruBlockCache org.apache.hadoop.hbase.client.TestMultiParallel org.apache.hadoop.hbase.master.TestRestartCluster org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.regionserver.TestStore org.apache.hadoop.hbase.regionserver.wal.TestHLogBench org.apache.hadoop.hbase.rest.TestGzipFilter org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD org.apache.hadoop.hbase.master.TestLogsCleaner org.apache.hadoop.hbase.regionserver.TestAtomicOperation org.apache.hadoop.hbase.rest.TestScannersWithFilters org.apache.hadoop.hbase.TestInfoServers org.apache.hadoop.hbase.regionserver.TestParallelPut org.apache.hadoop.hbase.coprocessor.TestClassLoading org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.filter.TestColumnRangeFilter org.apache.hadoop.hbase.mapred.TestTableInputFormat org.apache.hadoop.hbase.client.TestHCM org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole org.apache.hadoop.hbase.coprocessor.TestMasterObserver org.apache.hadoop.hbase.rest.TestStatusResource org.apache.hadoop.hbase.TestRegionRebalancing org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort org.apache.hadoop.hbase.regionserver.TestSeekOptimizations org.apache.hadoop.hbase.rest.TestVersionResource org.apache.hadoop.hbase.client.TestScannerTimeout org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol org.apache.hadoop.hbase.regionserver.TestSplitTransaction org.apache.hadoop.hbase.rest.TestRowResource org.apache.hadoop.hbase.rest.TestScannerResource org.apache.hadoop.hbase.ipc.TestDelayedRpc org.apache.hadoop.hbase.rest.client.TestRemoteAdmin org.apache.hadoop.hbase.util.TestFSUtils org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.rest.TestTableResource org.apache.hadoop.hbase.regionserver.wal.TestWALReplay org.apache.hadoop.hbase.master.TestHMasterRPCException org.apache.hadoop.hbase.util.TestIdLock org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster org.apache.hadoop.hbase.regionserver.TestMemStore org.apache.hadoop.hbase.rest.TestTransform org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit org.apache.hadoop.hbase.regionserver.TestHRegion org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13161285#comment-13161285 ] Hadoop QA commented on HBASE-4415: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505810/HBASE-4415-8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 71 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.client.TestInstantSchemaChange Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/419//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/419//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/419//console This message is automatically generated. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.92.1 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415-7.patch, HBASE-4415-8.patch, HBASE-4415-9.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13161317#comment-13161317 ] Hadoop QA commented on HBASE-4415: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505818/HBASE-4415-9.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 71 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.client.TestInstantSchemaChange Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/420//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/420//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/420//console This message is automatically generated. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.92.1 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415-7.patch, HBASE-4415-8.patch, HBASE-4415-9.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13151667#comment-13151667 ] Eric Yang commented on HBASE-4415: -- Stack, no problem. If there is any change required, let me know. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148627#comment-13148627 ] stack commented on HBASE-4415: -- @Eric I don't want to bring this set of patches into 0.92. Though this patch was submitted a good while ago, its only gotten attention from us lot recently -- thanks for keeping it up since original submit and sorry that you have had to kick us to take a look -- but I think that there needs to be more review before it goes in (I can help get more reviews and I'll dig in more after I get an RC 0.92 out). Its more of a new feature than it is a critical bug fix so I'm fine w/ putting it off. We'll can get it into trunk soon. Good on you Eric. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13144405#comment-13144405 ] Eric Yang commented on HBASE-4415: -- Ted, HBASE-4498 contains the explanation of reason to remove zookeeper-env.sh because ZooKeeper community do not support this in their RPM creation. Stack, hbase-env.sh in src/packages/templates/conf is used for hbase-setup-conf.sh as post installation process. It is designed to be a template, where conf/hbase-env.sh can be edited in place, which I think they should be separated to be safe. I am happy to convert it to use conf/hbase-env.sh location, but this would make conf/hbase-env.sh a template. Is this something worth proceeding? Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142284#comment-13142284 ] stack commented on HBASE-4415: -- Ditto on what Ted asks above. Why would we have this duplicated (and now behind) hbase-env.sh over in src/packages/templates/conf/hbase-env.sh? Why would we not copy it from original location? @Andrew I see this patch has: {code} + property +namehbase.master.kerberos.principal/name +value${HBASE_M_K_PRINCIPAL}/value +description/description + /property {code} We need more than that now? Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13141391#comment-13141391 ] Ted Yu commented on HBASE-4415: --- In update-hbase-env.sh: {code} -. /etc/default/zookeeper-env.sh {code} What's the rationale behind the above change ? Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13141407#comment-13141407 ] Andrew Purtell commented on HBASE-4415: --- For 0.92, this patch is missing the as-yet undocumented security configuration settings. They are in HBASE-2742 and HBASE-3025 at the moment, and are expected to go in to a 0.92 RC. I refer to HBase specific security features, not the Hadoop security integration, which this patch seems to cover. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133174#comment-13133174 ] Eric Yang commented on HBASE-4415: -- Failed core tests, and javadoc warnings don't seem to be related to this patch. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132076#comment-13132076 ] Hadoop QA commented on HBASE-4415: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12499070/HBASE-4415-6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6//console This message is automatically generated. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132205#comment-13132205 ] Hadoop QA commented on HBASE-4415: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12499070/HBASE-4415-6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 8 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.regionserver.wal.TestLogRolling Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7//console This message is automatically generated. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Affects Versions: 0.90.4, 0.92.0 Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Fix For: 0.90.4, 0.92.0 Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415-5.patch, HBASE-4415-6.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13115085#comment-13115085 ] Giridharan Kesavan commented on HBASE-4415: --- Eric, Could you please add the following to the hbaese-setup-conf script? set keytab directory permission to 700 set the service keytab owner to hbaseuser Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113727#comment-13113727 ] Giridharan Kesavan commented on HBASE-4415: --- can we have the option to set diff keytabs for hbasemaster and region servers? Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113737#comment-13113737 ] Eric Yang commented on HBASE-4415: -- Updated the setup script to use unique key tabs for HBase master and region servers. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113742#comment-13113742 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/ --- (Updated 2011-09-23 20:22:38.162933) Review request for hbase. Changes --- Updated the ability to use different keytab for HBase master and region server. Summary --- Create a post installation script to streamline configuration tasks for HBase. usage: /usr/sbin/hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location This addresses bug HBASE-4415. https://issues.apache.org/jira/browse/HBASE-4415 Diffs (updated) - /src/assembly/all.xml 1172936 /src/docbkx/getting_started.xml 1172936 /src/packages/hbase-setup-conf.sh PRE-CREATION /src/packages/templates/conf/hbase-env.sh PRE-CREATION /src/packages/templates/conf/hbase-site.xml PRE-CREATION Diff: https://reviews.apache.org/r/1925/diff Testing --- Thanks, Eric Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113791#comment-13113791 ] Giridharan Kesavan commented on HBASE-4415: --- hbase-setup-conf.sh script check for $HADOOP_HOME/bin/hadoop and looks for $HADOOP_HOME/hadoop-core*.jar when HADOOP_HOME is set to /usr the first condition passes and the second condition fails for not being able to find the hadoop-core jar as the core jar is installed by hadoop rpm at /usr/share/hadoop/hadoop-core-*.jar Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113828#comment-13113828 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/ --- (Updated 2011-09-23 23:17:00.742946) Review request for hbase. Changes --- Patch for finding location of hadoop jar files. Summary --- Create a post installation script to streamline configuration tasks for HBase. usage: /usr/sbin/hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location This addresses bug HBASE-4415. https://issues.apache.org/jira/browse/HBASE-4415 Diffs (updated) - /src/assembly/all.xml 1175049 /src/docbkx/getting_started.xml 1175049 /src/packages/hbase-setup-conf.sh PRE-CREATION /src/packages/templates/conf/hbase-env.sh PRE-CREATION /src/packages/templates/conf/hbase-site.xml PRE-CREATION Diff: https://reviews.apache.org/r/1925/diff Testing --- Thanks, Eric Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415-3.patch, HBASE-4415-4.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107923#comment-13107923 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- bq. On 2011-09-18 04:41:44, Lars Hofhansl wrote: bq. Generally it might be good to be able to define prefixes similar to what autoconf scripts do. Such as: bq. --prefix=DIR[PREFIX/etc] bq. --sysconfdir=DIR[PREFIX/etc] bq. bq. And so on. bq. bq. The hbase-site.xml and hbase-env.sh templates now have to be maintained separately, hence this makes config changes harder to maintain. bq. bq. In general this seems to be mostly useful for beginners of HBase, right? bq. (Oldtimers will have a mechanism in place to setup HBase - puppet, chef, fabric, etc. bq. bq. So I wonder if a more useful avenue would be to post this as part of the HBase book or some other getting started spot on the HBase site. bq. Hadoop related software used to deploy in isolated directory and use environment variables to drive the location of the software. GNU layout of single prefix does not work for tarball deployment because there are prefix directory per service. This is the reason that --hadoop-home, --hbase-home are implemented instead of --prefix. This is both useful for beginner or for script automation. I will include some documents for HBase book or HBase site. bq. On 2011-09-18 04:41:44, Lars Hofhansl wrote: bq. /src/packages/hbase-setup-conf.sh, line 177 bq. https://reviews.apache.org/r/1925/diff/1/?file=41428#file41428line177 bq. bq. Isn't /usr a strange default for a Hadoop (or HBase or ZooKeeper) installation? bq. Maybe the script should rather check for a value, and fail if none was given. Existing tarball deployment does not have a default location, hence, the default is set to /usr for rpm/debian package default location. I will added a check for /usr/bin/hadoop and bail out if hadoop does not exist. bq. On 2011-09-18 04:41:44, Lars Hofhansl wrote: bq. /src/packages/hbase-setup-conf.sh, line 245 bq. https://reviews.apache.org/r/1925/diff/1/?file=41428#file41428line245 bq. bq. I don't see HBASE_LOG_DIR defaulted anywhere. bq. Same for HBASE_PID_DIR. Good catch, I will make defaults for those two properties. How about default HBASE_LOG_DIR to ${HBASE_HOME}/var/log and ${HBASE_HOME}/var/run for consistency? bq. On 2011-09-18 04:41:44, Lars Hofhansl wrote: bq. /src/packages/templates/conf/hbase-env.sh, line 3 bq. https://reviews.apache.org/r/1925/diff/1/?file=41429#file41429line3 bq. bq. Wow 2007... that's old :) bq. I think the latest convention is not to include he year. I will make the change to exclude the year. Thanks :) - Eric --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/#review1957 --- On 2011-09-16 00:06:21, Eric Yang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1925/ bq. --- bq. bq. (Updated 2011-09-16 00:06:21) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Create a post installation script to streamline configuration tasks for HBase. bq. bq. usage: /usr/sbin/hbase-setup-conf.sh parameters bq. bq.Optional parameters: bq. --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location bq. --hadoop-home=/usr Set Hadoop directory location bq. --hadoop-namenode=localhost Set Hadoop namenode hostname bq. --hadoop-replication=3 Set HDFS replication bq. --hbase-home=/usrSet HBase directory location bq. --hbase-conf=/etc/hbase Set HBase configuration directory location bq. --hbase-log=/var/log/hbase Set HBase log directory location bq. --hbase-pid=/var/run/hbase Set HBase pid directory location bq. --hbase-user=hbase Set HBase user bq. --java-home=/usr/java/defaultSet JAVA_HOME directory location bq. --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm bq. --kerberos-principal-id=_HOSTSet Kerberos principal ID bq. --keytab-dir=/etc/security/keytabs Set keytab directory bq. --regionservers=localhostSet regionservers hostnames bq. --zookeeper-home=/usrSet ZooKeeper directory location bq. --zookeeper-quorum=localhost Set ZooKeeper Quorum bq.
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108306#comment-13108306 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/ --- (Updated 2011-09-20 01:17:07.234363) Review request for hbase. Changes --- - Added check for Hadoop installation. - Added default locations for HBase log/pid directories. - Added instruction to run hbase-setup-conf.sh in HBase Book. - Removed copyright date from license file. Summary --- Create a post installation script to streamline configuration tasks for HBase. usage: /usr/sbin/hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location This addresses bug HBASE-4415. https://issues.apache.org/jira/browse/HBASE-4415 Diffs (updated) - /src/assembly/all.xml 1170899 /src/docbkx/getting_started.xml 1172936 /src/packages/hbase-setup-conf.sh PRE-CREATION /src/packages/templates/conf/hbase-env.sh PRE-CREATION /src/packages/templates/conf/hbase-site.xml PRE-CREATION Diff: https://reviews.apache.org/r/1925/diff Testing --- Thanks, Eric Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108308#comment-13108308 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/ --- (Updated 2011-09-20 01:20:32.140897) Review request for hbase. Changes --- Made a mistake in previous diff. Upload the same patch without specifying parent patch. - Added check for Hadoop installation. - Added default locations for HBase log/pid directories. - Added instruction to run hbase-setup-conf.sh in HBase Book. - Removed copyright date from license file. Summary --- Create a post installation script to streamline configuration tasks for HBase. usage: /usr/sbin/hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location This addresses bug HBASE-4415. https://issues.apache.org/jira/browse/HBASE-4415 Diffs (updated) - /src/assembly/all.xml 1172936 /src/docbkx/getting_started.xml 1172936 /src/packages/hbase-setup-conf.sh PRE-CREATION /src/packages/templates/conf/hbase-env.sh PRE-CREATION /src/packages/templates/conf/hbase-site.xml PRE-CREATION Diff: https://reviews.apache.org/r/1925/diff Testing --- Thanks, Eric Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415-2.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107509#comment-13107509 ] stack commented on HBASE-4415: -- Will the below break what we have now? {code} fileSet + directorysrc/packages/templates/conf/directory + outputDirectoryshare/hbase/templates/conf/outputDirectory +/fileSet +fileSet {code} Do we have to reproduce hbase-env.sh over in src/packages? Similar for hbase-site.xml. Can you point me at why src/packages. Is this from hadoop and its src for building rpms etc.? So, how would i use this new script? Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107552#comment-13107552 ] Eric Yang commented on HBASE-4415: -- If no kerberos information is provided, the script will automatically setup as non-secure cluster. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107551#comment-13107551 ] Eric Yang commented on HBASE-4415: -- *share/hbase/templates/conf* is an newly added template directory. People can still use the default files in conf directory for backward compatibility until hbase-setup-conf.sh become the standard method of configuring HBase cluster. For now, there are two copies of hbase-env.sh, one in conf directory, and one in src/packages/templates/conf. The copy in src/packages/templates/conf is designed to be consumed by hbase-setup-conf.sh, and it mimics the changes that happens to trunk/conf/hbase-env.sh. If the community agrees that we can use one copy of base-env.sh and hbase-site.xml for both purpose of script automation and hand edit config file. I am more than willing to unify them as one copy and place in src/packages/templates/conf for clarity to indicate this is for script automation. Standard Maven practice uses the following structure: {noformat} src/main - application code src/test - test code {noformat} In Hadoop related projects, release packaging has introduced new directory structure: {noformat} src/packages/tarball - tarball manifest file or related meta data src/packages/rpm - rpm related meta data src/packages/deb - debian related meta data src/packages/template/conf - Configuration template files {noformat} This practice is to improve the release packages. In the past, Hadoop related projects are releasing a binary/source hybrid tar ball. This has proven problematic to maintain for different architecture. The intention is to improve the system so that we can create binary release for each architecture and decouple source and binary from tar ball releases. There are some more work to be done for packaging separation, but it is not in scope for this jira. The script is designed to run as root. For setting up a hbase cluster with hadoop 0.20.205. There are a few assumption made for kerberos key tabs. The key tab file names should be: namenode - nn.service.keytab secondarynamenode - sn.service.keytab datanode - dn.service.keytab jobtracker- jt.service.keytab tasktracker - tt.service.keytab hbase - hbase.service.keytab Compose a shell script that invokes each component's installation script and run them through pdsh, or use Apache Ambari (HMS): {noformat} export namenode=nn1 export datanode=dn1,dn2,dn3,dn4 export secondary_namenode=sn1 export jobtracker=jt1 export tasktracker=tt1,tt2,tt3,tt4 export hbase_regionservers=rs1,rs2,rs3,rs4 export hbase_master=hbm # Setup Hadoop configuration /home/hms/apps/hadoop-0.20.205.0/sbin/hadoop-setup-conf.sh \ --java-home=/home/hms/apps/jdk1.6.0_27 \ --namenode-host=${namenode} \ --secondarynamenode-host=${secondary_namenode} \ --jobtracker-host=${jobtracker} \ --conf-dir=/etc/hadoop \ --hdfs-dir=/data/0/hadoop/var/hdfs,/data/1/hadoop/var/hdfs,/data/2/hadoop/var/hdfs,/data/3/hadoop/var/hdfs \ --namenode-dir=/data/2/hadoop/var/hdfs/name \ --mapred-dir=/data/0/tmp/mapred-local,/data/1/tmp/mapred-local,/data/2/tmp/mapred-local,/data/3/tmp/mapred-local \ --datanode-dir=/data/0/hadoop/var/hdfs/data,/data/1/hadoop/var/hdfs/data,/data/2/hadoop/var/hdfs/data,/data/3/hadoop/var/hdfs/data \ --log-dir=/home/hms/apps/hadoop-0.20.205.0/logs \ --pid-dir=/home/hms/apps/hadoop-0.20.205.0/logs \ --hdfs-user=hdfs \ --mapreduce-user=mr \ --kerberos-realm=APACHE.ORG \ --hdfs-user-keytab=/home/hdfs/hdfs.headless.keytab \ --mapreduce-user-keytab=/home/mr/mr.headless.keytab \ --datanodes=${datanode} \ --tasktrackers=${tasktracker} \ --group=hadoop \ --auto # Format Hadoop HDFS /home/hms/apps/hadoop-0.20.205.0/sbin/hadoop-setup-hdfs.sh \ --config /etc/hadoop \ --hdfs-user=hrt_hdfs \ --mapreduce-user=hrt_mr \ --kerberos-realm=APACHE.ORG \ --hdfs-user-keytab=/home/hdfs/hdfs.headless.keytab \ --format # Start up secondary name node, datanodes, job tracker, task tracker through hadoop-daemon.sh # Setup HBase /home/hms/apps/hbase-0.90.3/sbin/hbase-setup-conf.sh \ --hadoop-conf=/etc/hadoop \ --hadoop-home=/home/hms/apps/hadoop-0.20.205.0 \ --hadoop-namenode=${namenode} \ --hbase-home=/home/hms/apps/hbase-0.90.3 \ --hbase-conf=/home/hms/apps/hbase-0.90.3/conf \ --hbase-log=/home/hms/apps/hbase-0.90.3/logs \ --hbase-pid=/home/hms/apps/hbase-0.90.3/var/run \ --java-home=/usr/java/default \ --kerberos-realm=APACHE.ORG \ --kerberos-principal-id=hbase/_HOST \ --keytab-dir=/etc/security/keytabs \ --regionservers=${hbase_regionservers} \ --zookeeper-quorum=${hbase_master} \ --zookeeper-snapshot=/home/hms/apps/var/lib/zookeeper --hbase-user=hbase --hadoop-replication=3 # Start HBase Master, region servers using hbase-daemon.sh. {noformat} Hope this helps. Add configuration script for
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107341#comment-13107341 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/#review1957 --- Generally it might be good to be able to define prefixes similar to what autoconf scripts do. Such as: --prefix=DIR[PREFIX/etc] --sysconfdir=DIR[PREFIX/etc] And so on. The hbase-site.xml and hbase-env.sh templates now have to be maintained separately, hence this makes config changes harder to maintain. In general this seems to be mostly useful for beginners of HBase, right? (Oldtimers will have a mechanism in place to setup HBase - puppet, chef, fabric, etc. So I wonder if a more useful avenue would be to post this as part of the HBase book or some other getting started spot on the HBase site. /src/packages/hbase-setup-conf.sh https://reviews.apache.org/r/1925/#comment4436 Isn't /usr a strange default for a Hadoop (or HBase or ZooKeeper) installation? Maybe the script should rather check for a value, and fail if none was given. /src/packages/hbase-setup-conf.sh https://reviews.apache.org/r/1925/#comment4437 I don't see HBASE_LOG_DIR defaulted anywhere. Same for HBASE_PID_DIR. /src/packages/templates/conf/hbase-env.sh https://reviews.apache.org/r/1925/#comment4438 Wow 2007... that's old :) I think the latest convention is not to include he year. - Lars On 2011-09-16 00:06:21, Eric Yang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1925/ bq. --- bq. bq. (Updated 2011-09-16 00:06:21) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Create a post installation script to streamline configuration tasks for HBase. bq. bq. usage: /usr/sbin/hbase-setup-conf.sh parameters bq. bq.Optional parameters: bq. --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location bq. --hadoop-home=/usr Set Hadoop directory location bq. --hadoop-namenode=localhost Set Hadoop namenode hostname bq. --hadoop-replication=3 Set HDFS replication bq. --hbase-home=/usrSet HBase directory location bq. --hbase-conf=/etc/hbase Set HBase configuration directory location bq. --hbase-log=/var/log/hbase Set HBase log directory location bq. --hbase-pid=/var/run/hbase Set HBase pid directory location bq. --hbase-user=hbase Set HBase user bq. --java-home=/usr/java/defaultSet JAVA_HOME directory location bq. --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm bq. --kerberos-principal-id=_HOSTSet Kerberos principal ID bq. --keytab-dir=/etc/security/keytabs Set keytab directory bq. --regionservers=localhostSet regionservers hostnames bq. --zookeeper-home=/usrSet ZooKeeper directory location bq. --zookeeper-quorum=localhost Set ZooKeeper Quorum bq. --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location bq. bq. bq. This addresses bug HBASE-4415. bq. https://issues.apache.org/jira/browse/HBASE-4415 bq. bq. bq. Diffs bq. - bq. bq./src/assembly/all.xml 1170899 bq./src/packages/hbase-setup-conf.sh PRE-CREATION bq./src/packages/templates/conf/hbase-env.sh PRE-CREATION bq./src/packages/templates/conf/hbase-site.xml PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/1925/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Eric bq. bq. Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105784#comment-13105784 ] Eric Yang commented on HBASE-4415: -- Review Board URL: https://reviews.apache.org/r/1925/ Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105785#comment-13105785 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/ --- Review request for hbase. Summary --- Create a post installation script to streamline configuration tasks for HBase. usage: /usr/sbin/hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location This addresses bug HBASE-4415. https://issues.apache.org/jira/browse/HBASE-4415 Diffs - /src/assembly/all.xml 1170899 /src/packages/hbase-setup-conf.sh PRE-CREATION /src/packages/templates/conf/hbase-env.sh PRE-CREATION /src/packages/templates/conf/hbase-site.xml PRE-CREATION Diff: https://reviews.apache.org/r/1925/diff Testing --- Thanks, Eric Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4415) Add configuration script for setup HBase (hbase-setup-conf.sh)
[ https://issues.apache.org/jira/browse/HBASE-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105786#comment-13105786 ] jirapos...@reviews.apache.org commented on HBASE-4415: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1925/ --- (Updated 2011-09-16 00:06:21.722559) Review request for hbase. Summary (updated) --- Create a post installation script to streamline configuration tasks for HBase. usage: /usr/sbin/hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location This addresses bug HBASE-4415. https://issues.apache.org/jira/browse/HBASE-4415 Diffs - /src/assembly/all.xml 1170899 /src/packages/hbase-setup-conf.sh PRE-CREATION /src/packages/templates/conf/hbase-env.sh PRE-CREATION /src/packages/templates/conf/hbase-site.xml PRE-CREATION Diff: https://reviews.apache.org/r/1925/diff Testing --- Thanks, Eric Add configuration script for setup HBase (hbase-setup-conf.sh) -- Key: HBASE-4415 URL: https://issues.apache.org/jira/browse/HBASE-4415 Project: HBase Issue Type: New Feature Components: scripts Environment: Java 6, Linux Reporter: Eric Yang Assignee: Eric Yang Attachments: HBASE-4415-1.patch, HBASE-4415.patch The goal of this jura is to provide a installation script for configuring HBase environment and configuration. By using the same pattern of *-setup-conf.sh for all Hadoop related projects. For HBase, the usage of the script looks like this: {noformat} usage: ./hbase-setup-conf.sh parameters Optional parameters: --hadoop-conf=/etc/hadoopSet Hadoop configuration directory location --hadoop-home=/usr Set Hadoop directory location --hadoop-namenode=localhost Set Hadoop namenode hostname --hadoop-replication=3 Set HDFS replication --hbase-home=/usrSet HBase directory location --hbase-conf=/etc/hbase Set HBase configuration directory location --hbase-log=/var/log/hbase Set HBase log directory location --hbase-pid=/var/run/hbase Set HBase pid directory location --hbase-user=hbase Set HBase user --java-home=/usr/java/defaultSet JAVA_HOME directory location --kerberos-realm=KERBEROS.EXAMPLE.COMSet Kerberos realm --kerberos-principal-id=_HOSTSet Kerberos principal ID --keytab-dir=/etc/security/keytabs Set keytab directory --regionservers=localhostSet regionservers hostnames --zookeeper-home=/usrSet ZooKeeper directory location --zookeeper-quorum=localhost Set ZooKeeper Quorum --zookeeper-snapshot=/var/lib/zookeeper Set ZooKeeper snapshot location {noformat} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira