Repository: kylin Updated Branches: refs/heads/document 84c961a39 -> 0e6aaa2f2
refine blog aws emr Signed-off-by: lidongsjtu <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/kylin/repo Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/0e6aaa2f Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/0e6aaa2f Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/0e6aaa2f Branch: refs/heads/document Commit: 0e6aaa2f2a3b258e905623d5c655b3abaaf9ce07 Parents: 84c961a Author: Roger Shi <[email protected]> Authored: Tue May 24 16:55:13 2016 +0800 Committer: lidongsjtu <[email protected]> Committed: Tue May 24 17:10:04 2016 +0800 ---------------------------------------------------------------------- website/_posts/blog/2016-05-24-aws-emr.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kylin/blob/0e6aaa2f/website/_posts/blog/2016-05-24-aws-emr.md ---------------------------------------------------------------------- diff --git a/website/_posts/blog/2016-05-24-aws-emr.md b/website/_posts/blog/2016-05-24-aws-emr.md index cdfae19..560aacf 100644 --- a/website/_posts/blog/2016-05-24-aws-emr.md +++ b/website/_posts/blog/2016-05-24-aws-emr.md @@ -7,12 +7,12 @@ categories: blog --- # Install Apache Kylin on EMR # -1. Create an EMR cluster from AWS console, and remember to pick the applications configuration which contains HBase and Hive as shown here. +1. Create an EMR cluster from AWS console, and remember to pick the applications configuration which contains HBase and Hive.  2. Login to master node of the cluster with ssh. ([instruction](http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-connect-master-node-ssh.html)) -3. Download and install Kylin binary, we take Kylin version 1.5.1 as example here (Please refer to [Kylin download page](http://kylin.apache.org/download/) for latest binary package). Run the commands below. +3. Download and install Kylin binary, we take Kylin version 1.5.1 as example here (please refer to [Kylin download page](http://kylin.apache.org/download/) for latest binary package). Run the commands below. ``` wget https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz @@ -23,9 +23,9 @@ export KYLIN_HOME=`pwd` # Configure Apache Kylin -To make Kylin run well on AWS EMR we need to change some scripts. The fix can be found in this [commit](https://github.com/apache/kylin/commit/dc08186d570e16b37d9ddaab80aba28801cdb3d0) and it'll be available in Kylin's future version. +To make Kylin run well on AWS EMR we need to modify some scripts. The fix can be found in [this commit](https://github.com/apache/kylin/commit/dc08186d570e16b37d9ddaab80aba28801cdb3d0) and it'll be available in Kylin's future version. -*First file is $KYLIN_HOME/bin/find-hive-dependency.sh, starting from line 68. +First file is `$KYLIN_HOME/bin/find-hive-dependency.sh`, starting from line 68. ``` 68 if [ -z "$HCAT_HOME" ] @@ -40,7 +40,7 @@ To make Kylin run well on AWS EMR we need to change some scripts. The fix can be + elif [ -n is_aws ] && [ -d "/usr/lib/oozie/lib" ]; then + hcatalog_home=/usr/lib/oozie/lib 76 else - 77 echo "Couldn't locate hcatalog installation, please make sure it is installed and set HCAT_HOME to the path." + 77 echo "Couldn't locate hcatalog installation, please make sure it is installed and set HCAT_HOME..." 78 exit 1 79 fi 80 else @@ -49,9 +49,9 @@ To make Kylin run well on AWS EMR we need to change some scripts. The fix can be 83 fi ``` -When Kylin starts running it tries to detect environment such as Hive home dictionary. It goes well most time but not for AWS EMR. In AWS EMR nodes, Hcatalog libs is under "/usr/lib/oozie/lib", which is not expected. The modification makes Kylin handle it specially once detecting it's running on AWS EMR. +Kylin tries to detect environment such as Hive home dictionary on starting. It goes well in most time but not in AWS EMR case. In AWS EMR nodes Hcatalog libs is under `/usr/lib/oozie/lib`, which is not expected. The modification makes Kylin handle it specially once detecting it's on AWS EMR. -*Second file is $KYLIN_HOME/bin/find-hbase-dependency.sh, starting from line 20. +Second file is `$KYLIN_HOME/bin/find-hbase-dependency.sh`, starting from line 20. ``` 20 hbase_classpath=`hbase classpath` @@ -64,7 +64,7 @@ When Kylin starts running it tries to detect environment such as Hive home dicti 21 arr=(`echo $hbase_classpath | cut -d ":" --output-delimiter=" " -f 1-`) ``` -In AWS EMR, Hbase scripts reset environment variable HBASE_CLASSPATH in its first run, which is determined by environment variable HBASE_ENV_INIT. Kylin builds its own class path according to this variable. To avoid missing libs caused by HBASE_CLASSPATH reseting, Kylin set HBASE_ENV_INIT to "true" after running "hbase classpath" so that HBASE_CLASSPATH won't be reset next time run "hbase classpath". +In AWS EMR Hbase scripts reset environment variable `HBASE_CLASSPATH` in its first run, which is determined by environment variable `HBASE_ENV_INIT`. Kylin builds its own class path according to this variable. To avoid missing libs caused by `HBASE_CLASSPATH` reseting, Kylin set `HBASE_ENV_INIT` to `"true"` so that `HBASE_CLASSPATH` won't be reset next time run `hbase classpath`. # Load sample data and start Apache Kylin
