[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313360#comment-15313360
]
Andrew Purtell commented on HBASE-12721:
----------------------------------------
bq. clusterdock_run ./bin/build_cluster apache_hbase --hbase-version=0.98
--hbase-git-commit=0.98
Let me try this now. Really liking the ability to specify any git ref.
bq. Once those images exist, you can pass the location of an .ini-like file as
an argument to start_cluster and it will set configurations for you before
starting services
Ok, what I would like to do is control how hadoop-env.sh and hbase-env.sh are
generated. Consider something like:
*hadoop-env.sh*
{noformat}
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+UseG1GC"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:MaxGCPauseMillis=100"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+PrintGCDetails"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+PrintGCDateStamps"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+PrintGCTimeStamps"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+PrintAdaptiveSizePolicy"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+PrintReferenceGC"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+ParallelRefProcEnabled"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:+TieredCompilation"
COMMON_HDFS_OPTS="\$COMMON_HDFS_OPTS -XX:-ResizePLAB"
export HADOOP_NAMENODE_OPTS="\$HADOOP_NAMENODE_OPTS -Xms1g -Xmx1g"
export HADOOP_NAMENODE_OPTS="\$HADOOP_NAMENODE_OPTS \$COMMON_HDFS_OPTS"
export HADOOP_NAMENODE_OPTS="\$HADOOP_NAMENODE_OPTS -XX:+AlwaysPreTouch"
export HADOOP_NAMENODE_OPTS="\$HADOOP_NAMENODE_OPTS -verbose:gc
-Xloggc:/var/log/hadoop/hdfs-namenode-gc.log"
export HADOOP_SECONDARYNAMENODE_OPTS="\$HADOOP_SECONDARYNAMENODE_OPTS -Xms1g
-Xmx1g"
export HADOOP_SECONDARYNAMENODE_OPTS="\$HADOOP_SECONDARYNAMENODE_OPTS
\$COMMON_HDFS_OPTS"
export HADOOP_SECONDARYNAMENODE_OPTS="\$HADOOP_SECONDARYNAMENODE_OPTS
-verbose:gc -Xloggc:/var/log/hadoop/hdfs-secondarynamenode-gc.log"
export HADOOP_DATANODE_OPTS="\$HADOOP_DATANODE_OPTS -Xms1g -Xmx1g"
export HADOOP_DATANODE_OPTS="\$HADOOP_DATANODE_OPTS \$COMMON_HDFS_OPTS"
export HADOOP_DATANODE_OPTS="\$HADOOP_DATANODE_OPTS -XX:+AlwaysPreTouch"
export HADOOP_DATANODE_OPTS="\$HADOOP_DATANODE_OPTS -verbose:gc
-Xloggc:/var/log/hadoop/hdfs-datanode-gc.log"
{noformat}
*hbase-env.sh*
{noformat}
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+UseG1GC"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+PrintGCDetails"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+PrintGCDateStamps"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+PrintGCTimeStamps"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+PrintAdaptiveSizePolicy"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+PrintReferenceGC"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+ParallelRefProcEnabled"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:+TieredCompilation"
COMMON_HBASE_OPTS="\$COMMON_HBASE_OPTS -XX:-ResizePLAB"
export HBASE_MASTER_OPTS="\$HBASE_MASTER_OPTS -Xms1g -Xmx1g"
export HBASE_MASTER_OPTS="\$HBASE_MASTER_OPTS \$COMMON_HBASE_OPTS"
export HBASE_MASTER_OPTS="\$HBASE_MASTER_OPTS -verbose:gc
-Xloggc:/var/log/hbase/hbase-master-gc.log"
export HBASE_REGIONSERVER_OPTS="\$HBASE_REGIONSERVER_OPTS -Xms32g -Xmx32g"
export HBASE_REGIONSERVER_OPTS="\$HBASE_REGIONSERVER_OPTS \$COMMON_HBASE_OPTS"
export HBASE_REGIONSERVER_OPTS="\$HBASE_REGIONSERVER_OPTS
-XX:MaxGCPauseMillis=50"
export HBASE_REGIONSERVER_OPTS="\$HBASE_REGIONSERVER_OPTS -XX:+UseCondCardMark"
export HBASE_REGIONSERVER_OPTS="\$HBASE_REGIONSERVER_OPTS -XX:+AlwaysPreTouch"
export HBASE_REGIONSERVER_OPTS="\$HBASE_REGIONSERVER_OPTS -verbose:gc
-Xloggc:/var/log/hbase/hbase-regionserver-gc.log"
export HBASE_ZOOKEEPER_OPTS="\$HBASE_ZOOKEEPER_OPTS -Xms1g -Xmx1g"
export HBASE_ZOOKEEPER_OPTS="\$HBASE_ZOOKEEPER_OPTS \$COMMON_HBASE_OPTS"
export HBASE_ZOOKEEPER_OPTS="\$HBASE_ZOOKEEPER_OPTS -XX:+AlwaysPreTouch"
export HBASE_ZOOKEEPER_OPTS="\$HBASE_ZOOKEEPER_OPTS -verbose:gc
-Xloggc:/var/log/hbase/hbase-zookeeper-gc.log"
{noformat}
That would handle all of the JVM flag wishlist I think.
bq. ./bin/build_cluster apache_hbase supports a flag that specifies a Java
tarball to use
Cool, I will try it.
bq. I imagine I'd need to modify the code a little bit to handle non-Oracle
releases, though.
Maybe not? The OpenJDK based JVM tarballs I build unpack like Oracle releases
and run fine on CentOS 6. They just lack JFR. I don't think you're using that.
> Create Docker container cluster infrastructure to enable better testing
> -----------------------------------------------------------------------
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Reporter: Dima Spivak
> Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
