Repository: kylin Updated Branches: refs/heads/document 0e6aaa2f2 -> 89d3bd0e4
Revert "refine blog aws emr" This reverts commit 0e6aaa2f2a3b258e905623d5c655b3abaaf9ce07. Project: http://git-wip-us.apache.org/repos/asf/kylin/repo Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/35634ce0 Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/35634ce0 Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/35634ce0 Branch: refs/heads/document Commit: 35634ce0c92483e99f0598aa11821b11d8aa8c39 Parents: 0e6aaa2 Author: lidongsjtu <[email protected]> Authored: Wed May 25 10:38:04 2016 +0800 Committer: lidongsjtu <[email protected]> Committed: Wed May 25 10:38:04 2016 +0800 ---------------------------------------------------------------------- website/_posts/blog/2016-05-24-aws-emr.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kylin/blob/35634ce0/website/_posts/blog/2016-05-24-aws-emr.md ---------------------------------------------------------------------- diff --git a/website/_posts/blog/2016-05-24-aws-emr.md b/website/_posts/blog/2016-05-24-aws-emr.md index 560aacf..cdfae19 100644 --- a/website/_posts/blog/2016-05-24-aws-emr.md +++ b/website/_posts/blog/2016-05-24-aws-emr.md @@ -7,12 +7,12 @@ categories: blog --- # Install Apache Kylin on EMR # -1. Create an EMR cluster from AWS console, and remember to pick the applications configuration which contains HBase and Hive. +1. Create an EMR cluster from AWS console, and remember to pick the applications configuration which contains HBase and Hive as shown here.  2. Login to master node of the cluster with ssh. ([instruction](http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-connect-master-node-ssh.html)) -3. Download and install Kylin binary, we take Kylin version 1.5.1 as example here (please refer to [Kylin download page](http://kylin.apache.org/download/) for latest binary package). Run the commands below. +3. Download and install Kylin binary, we take Kylin version 1.5.1 as example here (Please refer to [Kylin download page](http://kylin.apache.org/download/) for latest binary package). Run the commands below. ``` wget https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz @@ -23,9 +23,9 @@ export KYLIN_HOME=`pwd` # Configure Apache Kylin -To make Kylin run well on AWS EMR we need to modify some scripts. The fix can be found in [this commit](https://github.com/apache/kylin/commit/dc08186d570e16b37d9ddaab80aba28801cdb3d0) and it'll be available in Kylin's future version. +To make Kylin run well on AWS EMR we need to change some scripts. The fix can be found in this [commit](https://github.com/apache/kylin/commit/dc08186d570e16b37d9ddaab80aba28801cdb3d0) and it'll be available in Kylin's future version. -First file is `$KYLIN_HOME/bin/find-hive-dependency.sh`, starting from line 68. +*First file is $KYLIN_HOME/bin/find-hive-dependency.sh, starting from line 68. ``` 68 if [ -z "$HCAT_HOME" ] @@ -40,7 +40,7 @@ First file is `$KYLIN_HOME/bin/find-hive-dependency.sh`, starting from line 68. + elif [ -n is_aws ] && [ -d "/usr/lib/oozie/lib" ]; then + hcatalog_home=/usr/lib/oozie/lib 76 else - 77 echo "Couldn't locate hcatalog installation, please make sure it is installed and set HCAT_HOME..." + 77 echo "Couldn't locate hcatalog installation, please make sure it is installed and set HCAT_HOME to the path." 78 exit 1 79 fi 80 else @@ -49,9 +49,9 @@ First file is `$KYLIN_HOME/bin/find-hive-dependency.sh`, starting from line 68. 83 fi ``` -Kylin tries to detect environment such as Hive home dictionary on starting. It goes well in most time but not in AWS EMR case. In AWS EMR nodes Hcatalog libs is under `/usr/lib/oozie/lib`, which is not expected. The modification makes Kylin handle it specially once detecting it's on AWS EMR. +When Kylin starts running it tries to detect environment such as Hive home dictionary. It goes well most time but not for AWS EMR. In AWS EMR nodes, Hcatalog libs is under "/usr/lib/oozie/lib", which is not expected. The modification makes Kylin handle it specially once detecting it's running on AWS EMR. -Second file is `$KYLIN_HOME/bin/find-hbase-dependency.sh`, starting from line 20. +*Second file is $KYLIN_HOME/bin/find-hbase-dependency.sh, starting from line 20. ``` 20 hbase_classpath=`hbase classpath` @@ -64,7 +64,7 @@ Second file is `$KYLIN_HOME/bin/find-hbase-dependency.sh`, starting from line 20 21 arr=(`echo $hbase_classpath | cut -d ":" --output-delimiter=" " -f 1-`) ``` -In AWS EMR Hbase scripts reset environment variable `HBASE_CLASSPATH` in its first run, which is determined by environment variable `HBASE_ENV_INIT`. Kylin builds its own class path according to this variable. To avoid missing libs caused by `HBASE_CLASSPATH` reseting, Kylin set `HBASE_ENV_INIT` to `"true"` so that `HBASE_CLASSPATH` won't be reset next time run `hbase classpath`. +In AWS EMR, Hbase scripts reset environment variable HBASE_CLASSPATH in its first run, which is determined by environment variable HBASE_ENV_INIT. Kylin builds its own class path according to this variable. To avoid missing libs caused by HBASE_CLASSPATH reseting, Kylin set HBASE_ENV_INIT to "true" after running "hbase classpath" so that HBASE_CLASSPATH won't be reset next time run "hbase classpath". # Load sample data and start Apache Kylin
