Author: shaofengshi
Date: Fri Jan 8 03:52:56 2016
New Revision: 1723649
URL: http://svn.apache.org/viewvc?rev=1723649&view=rev
Log:
update howto_upgrade
Modified:
kylin/site/docs/howto/howto_upgrade.html
kylin/site/feed.xml
Modified: kylin/site/docs/howto/howto_upgrade.html
URL:
http://svn.apache.org/viewvc/kylin/site/docs/howto/howto_upgrade.html?rev=1723649&r1=1723648&r2=1723649&view=diff
==============================================================================
--- kylin/site/docs/howto/howto_upgrade.html (original)
+++ kylin/site/docs/howto/howto_upgrade.html Fri Jan 8 03:52:56 2016
@@ -1670,38 +1670,75 @@
<div id="pjax">
<h1
class="post-title">How to Upgrade</h1>
- <p>version:
v0.7.2, since: v0.7.1</p>
+ <p>version:
v1.2, since: v0.7.1</p>
<article
class="post-content" >
- <p>In v0.7, Kylin
refactored the metadata structure, for the new features like inverted-index and
streaming; If you have cube created with v0.6 and want to keep in v0.7, a
migration is needed; (Please skip v0.7.1 as<br />
-it has several compatible issues and the fix will be included in v0.7.2) Below
is the steps;</p>
+ <h2
id="upgrade-among-v07x-and-v1x">Upgrade among v0.7.x and v1.x</h2>
+
+<p>From v0.7.1 to latest v1.2, Kylinâs metadata is backward compatible, the
upgrade can be finished in couple of minutes:</p>
-<h1 id="backup-v06-metadata">Backup v0.6 metadata</h1>
+<h4 id="backup-metadata">1. Backup metadata</h4>
+<p>Backup the Kylin metadata peridically is a good practice, and is highly
suggested before upgrade;</p>
-<p>To avoid any data loss in the migration, a backup at the very beginning is
always suggested; You can use HBaseâs backup or snapshot command to achieve
this; Here is a sample with snapshot:</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>cd $KYLIN_HOME
+./bin/metastore.sh backup
+</code></pre>
+</div>
+<p>It will print the backup folder, take it down and make sure it will not be
deleted before the upgrade finished. If there is no âmetastore.shâ, use
HBaseâs snapshot command to do backup:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>hbase shell
snapshot 'kylin_metadata', 'kylin_metadata_backup20150610'
</code></pre>
</div>
+<p>Here âkylin_metadataâ is the default kylin metadata table name, replace
it with the right table name of your Kylin;</p>
-<p>âkylin_metadataâ is the default kylin metadata table name, replace it
with the right table name of your Kylin;</p>
+<h4 id="install-new-kylin-and-copy-back-conf">2. Install new Kylin and copy
back âconfâ</h4>
+<p>Download the new Kylin binary package from Kylinâs download page; Extract
it to a different folder other than current KYLIN_HOME; Before copy back the
âconfâ folder, do a compare and merge between the old and new
kylin.properties to ensure newly introduced property will be kept.</p>
+
+<h4 id="stop-old-and-start-new-kylin-instance">3. Stop old and start new Kylin
instance</h4>
+<div class="highlighter-rouge"><pre class="highlight"><code>cd $KYLIN_HOME
+./bin/kylin.sh stop
+export KYLIN_HOME="<path_of_new_installation>"
+cd $KYLIN_HOME
+./bin/kylin.sh start
+</code></pre>
+</div>
+
+<h4 id="back-port-if-the-upgrade-is-failed">4. Back-port if the upgrade is
failed</h4>
+<p>If the new version couldnât startup and need back-port, shutdown it and
then switch to the old KYLIN_HOME to start. Idealy that would return to the
origin state. If the metadata is broken, restore it from the backup folder.</p>
-<h1 id="dump-v06-metadata-to-local-file">Dump v0.6 metadata to local file</h1>
+<div class="highlighter-rouge"><pre class="highlight"><code>./bin/metastore.sh
restore <path_of_metadata_backup>
+</code></pre>
+</div>
+
+<h2 id="upgrade-from-v06x-to-v07x">Upgrade from v0.6.x to v0.7.x</h2>
+
+<p>In v0.7, Kylin refactored the metadata structure, for the new features like
inverted-index and streaming; If you have cube created with v0.6 and want to
keep in v0.7, a migration is needed; (Please skip v0.7.1 as<br />
+it has several compatible issues and the fix will be included in v0.7.2) Below
is the steps;</p>
+<h4 id="backup-v06-metadata">1. Backup v0.6 metadata</h4>
+<p>To avoid data loss in the migration, a backup at the very beginning is
always suggested; You can use HBaseâs backup or snapshot command to achieve
this; Here is a sample with snapshot:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>hbase shell
+snapshot 'kylin_metadata', 'kylin_metadata_backup20150610'
+</code></pre>
+</div>
+
+<p>âkylin_metadataâ is the default kylin metadata table name, replace it
with the right table name of your Kylin;</p>
+
+<h4 id="dump-v06-metadata-to-local-file">2. Dump v0.6 metadata to local
file</h4>
<p>This is also a backup method; As the migration tool is only tested with
local file system, this step is must; All metadata need be downloaded,
including snapshot, dictionary, etc;</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar
${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar
org.apache.kylin.common.persistence.ResourceTool download ./meta_dump
+<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-job.jar
org.apache.kylin.common.persistence.ResourceTool download ./meta_dump
</code></pre>
</div>
<p>(./meta_dump is the local folder that the metadata will be downloaded,
change to name you preferred)</p>
-<h1 id="run-cubemetadataupgrade-to-migrate-the-metadata">Run
CubeMetadataUpgrade to migrate the metadata</h1>
-
+<h4 id="run-cubemetadataupgrade-to-migrate-the-metadata">3. Run
CubeMetadataUpgrade to migrate the metadata</h4>
<p>This step is to run the migration tool to parse the v0.6 metadata and then
convert to v0.7 format; A verification will be performed in the last, and
report error if some cube couldnât be migrated;</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar
${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar
org.apache.kylin.job.CubeMetadataUpgrade ./meta_dump
+<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-job.jar
org.apache.kylin.job.CubeMetadataUpgrade ./meta_dump
</code></pre>
</div>
@@ -1709,27 +1746,25 @@ snapshot 'kylin_metadata', 'kylin_metada
<li>The tool will not overwrite v0.6 metadata; It will create a new folder
with â_v2â suffix in the same folder, in this case the â./meta_dump_v2â
will be created;</li>
<li>By default this tool will only migrate the job history in last 30 days;
If you want to keep elder job history, please tweak upgradeJobInstance() method
by your own;</li>
<li>If you see <em>No error or warning messages; The migration is
success</em> , thatâs good; Otherwise please check the error/warning messages
carefully;</li>
- <li>
- <p>For some problem you may need manually update the JSON file, to check
whether the problem is gone, you can run a verify against the new metadata:</p>
-
- <p>hbase org.apache.hadoop.util.RunJar
${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar
org.apache.kylin.job.CubeMetadataUpgrade ./meta_dump2 verify</p>
- </li>
+ <li>For some problem you may need manually update the JSON file, to check
whether the problem is gone, you can run a verify against the new metadata:</li>
</ol>
-<h1 id="upload-the-new-metadata-to-hbase">Upload the new metadata to HBase</h1>
+<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-job.jar
org.apache.kylin.job.CubeMetadataUpgrade ./meta_dump2 verify
+</code></pre>
+</div>
+<h4 id="upload-the-new-metadata-to-hbase">4. Upload the new metadata to
HBase</h4>
<p>Now the new format of metadata will be upload to the HBase to replace the
old format; Stop Kylin, and then:</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar
${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar
org.apache.kylin.common.persistence.ResourceTool reset
-hbase org.apache.hadoop.util.RunJar
${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar
org.apache.kylin.common.persistence.ResourceTool upload ./meta_dump_v2
+<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-job.jar
org.apache.kylin.common.persistence.ResourceTool reset
+hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-job.jar
org.apache.kylin.common.persistence.ResourceTool upload ./meta_dump_v2
</code></pre>
</div>
-<h1 id="update-htables-to-use-new-coprocessor">Update HTables to use new
coprocessor</h1>
-
+<h4 id="update-htables-to-use-new-coprocessor">5. Update HTables to use new
coprocessor</h4>
<p>Kylin uses HBase coprocessor to do server side aggregation; When Kylin
instance upgrades to V0.7, the HTables that created in V0.6 should also be
updated to use the new coprocessor:</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar
${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar
org.apache.kylin.job.tools.DeployCoprocessorCLI
${KYLIN_HOME}/lib/kylin-coprocessor-x.x.x-SNAPSHOT.jar
+<div class="highlighter-rouge"><pre class="highlight"><code>hbase
org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-job.jar
org.apache.kylin.job.tools.DeployCoprocessorCLI
${KYLIN_HOME}/lib/kylin-coprocessor-x.x.x.jar
</code></pre>
</div>
Modified: kylin/site/feed.xml
URL:
http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1723649&r1=1723648&r2=1723649&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Fri Jan 8 03:52:56 2016
@@ -19,8 +19,8 @@
<description>Apache Kylin Home</description>
<link>http://kylin.apache.org/</link>
<atom:link href="http://kylin.apache.org/feed.xml"; rel="self"
type="application/rss+xml"/>
- <pubDate>Wed, 06 Jan 2016 23:39:57 -0800</pubDate>
- <lastBuildDate>Wed, 06 Jan 2016 23:39:57 -0800</lastBuildDate>
+ <pubDate>Thu, 07 Jan 2016 19:52:21 -0800</pubDate>
+ <lastBuildDate>Thu, 07 Jan 2016 19:52:21 -0800</lastBuildDate>
<generator>Jekyll v2.5.3</generator>
<item>
