howto_backup_metadata.html feed.xml

lidong Fri, 22 Mar 2019 07:13:12 -0700

Author: lidong
Date: Fri Mar 22 14:12:46 2019
New Revision: 1856055

URL: http://svn.apache.org/viewvc?rev=1856055&view=rev
Log:
Update 2018-12-12-why-did-meituan-develop-kylin-on-druid-part1-of-2.md


Modified:
    kylin/site/blog/2015/12/25/support-powerbi-tableau9/index.html
    
kylin/site/blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/index.html
    kylin/site/cn/docs/howto/howto_backup_metadata.html
    kylin/site/docs/howto/howto_backup_metadata.html
    kylin/site/feed.xml

Modified: kylin/site/blog/2015/12/25/support-powerbi-tableau9/index.html
URL: 
http://svn.apache.org/viewvc/kylin/site/blog/2015/12/25/support-powerbi-tableau9/index.html?rev=1856055&r1=1856054&r2=1856055&view=diff
==============================================================================
--- kylin/site/blog/2015/12/25/support-powerbi-tableau9/index.html (original)
+++ kylin/site/blog/2015/12/25/support-powerbi-tableau9/index.html Fri Mar 22 
14:12:46 2019
@@ -6383,11 +6383,11 @@ var _hmt = _hmt || [];
 <p>Microsoft Power BI is a business intelligence tool providing rich 
functionality and experience for data visualization and processing to user.</p>
 
 <h3 id="tableau-9">Tableau 9</h3>
-<p>Tableau is one of the most famous business intelligence software on Windows 
platform. With simple operation and powerful functionality, users can easily 
drag and drop to visualize data in large scale. Besides, Tableau supports 
reading ODBC data sources, in a Live connection mode. In latest version 9.1, 
user experience gets better improved very much.</p>
+<p>Tableau is one of the most famous business intelligence software on Windows 
platform. With simple operation and powerful functionality, users can easily 
drag and drop to visualize data in large scale. Besides, Tableau supports 
reading ODBC data sources, in a Live connection mode. In latest version 9.1, 
user experience gets improved greatly.</p>
 
 <h3 id="support-from-kylin-community">Support from Kylin community</h3>
-<p>There are many requirements from eBay and other users who are using Kylin 
today, especially from analysts who using these tools everyday. The slow query 
performance of interactive with big data on Hadoop through other tool, like 
Hive, brings big chellenge for them.<br />
-With sub-seconds query latency feature of Apache Kylin platform, analysts are 
able to run interactive query agaisting billions rows data in seconds, with 
these fancy reporting and visualization tools.</p>
+<p>There are many requirements from eBay and other users who are using Kylin 
today, especially from analysts who uses these tools everyday. The slow query 
performance while interacting with big data on Hadoop through other tool, like 
Hive, brings big challenge for them.<br />
+With sub-seconds query latency feature of Apache Kylin platform, analysts are 
able to run interactive query against billions rows data in seconds, with these 
fancy reporting and visualization tools.</p>
 
 <h3 id="tutorial">Tutorial</h3>
 <p>Canât wait to try? Please refer to following tutorials:  <br />

Modified: 
kylin/site/blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/index.html
URL: 
http://svn.apache.org/viewvc/kylin/site/blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/index.html?rev=1856055&r1=1856054&r2=1856055&view=diff
==============================================================================
--- 
kylin/site/blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/index.html
 (original)
+++ 
kylin/site/blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/index.html
 Fri Mar 22 14:12:46 2019
@@ -6453,7 +6453,7 @@ Graphic 4 Druid Schema</p>
 
 <p>Kylin stores its data in HBase by converting the Dimensions and Measures 
into HBase Keys and Values, respectively. As HBase doesnât support secondary 
index and only has one RowKey index, Kylinâs Dimension values will be 
combined into a fixed sequence to store as RowKey. In this way, filtering on a 
Dimension in the front of the sequence will perform better than those at the 
back. Hereâs an example:</p>
 
-<p>In the testing environment, there are two almost identical Cubes (Cube1 and 
Cube2). They both have the same data source and the same Dimensions/Measures. 
The only difference is the order of the Dimensions in the RowKey: Cube1 puts 
P_LINEORDER.LO_CUSTKEY at the first while Cube2 the last.<br />
+<p>In the testing environment, there are two almost identical Cubes (Cube1 and 
Cube2). They both have the same data source and the same Dimensions/Measures. 
The only difference is the order of the Dimensions in the RowKey: Cube1 puts 
P_LINEORDER.LO_ORDERKEY at the first while Cube2 the last.<br />
 <img src="/images/blog/Kylin-On-Durid/5 cube1_rowkey_sequence.png" alt="" 
/><br />
 Graphic 5 Cube1 RowKey Sequence</p>
 

Modified: kylin/site/cn/docs/howto/howto_backup_metadata.html
URL: 
http://svn.apache.org/viewvc/kylin/site/cn/docs/howto/howto_backup_metadata.html?rev=1856055&r1=1856054&r2=1856055&view=diff
==============================================================================
--- kylin/site/cn/docs/howto/howto_backup_metadata.html (original)
+++ kylin/site/cn/docs/howto/howto_backup_metadata.html Fri Mar 22 14:12:46 2019
@@ -173,6 +173,104 @@ kylin.metadata.url=kylin_metadata@hbase<
 
 <p>è¿è¡¨æåæ°æ®ä¼è¢«ä¿åå¨ä¸ä¸ªå«ä½ âkylin_metadataâçhtable 
éãä½ å¯ä»¥å¨ hbase shell é scan è¯¥ htbale æ¥è·åå®ã</p>
 
+<h2 id="section">åæ°æ®è·¯å¾</h2>
+
+<p>Kylinä½¿ç¨<code class="highlighter-rouge">resource root path + resource 
name + resource suffix</code>ä½ä¸ºkeyå¼(HBaseä¸çrowkey)æ¥åå¨å
æ°æ®ãä½ å¯ä»¥åèå¦ä¸è¡¨æ ¼ä½¿ç¨<code 
class="highlighter-rouge">./bin/metastore.sh</code>å½ä»¤ã</p>
+
+<table>
+  <thead>
+    <tr>
+      <th>Resource root path</th>
+      <th style="text-align: left">resource name</th>
+      <th style="text-align: left">resource suffix</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>/cube</td>
+      <td style="text-align: left">/cube name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/cube_desc</td>
+      <td style="text-align: left">/cube name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/cube_statistics</td>
+      <td style="text-align: left">/cube name/uuid</td>
+      <td style="text-align: left">.seq</td>
+    </tr>
+    <tr>
+      <td>/model_desc</td>
+      <td style="text-align: left">/model name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/dict</td>
+      <td style="text-align: left">/DATABASE.TABLE/COLUMN/uuid</td>
+      <td style="text-align: left">.dict</td>
+    </tr>
+    <tr>
+      <td>/project</td>
+      <td style="text-align: left">/project name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/table_snapshot</td>
+      <td style="text-align: left">/DATABASE.TABLE/uuid</td>
+      <td style="text-align: left">.snapshot</td>
+    </tr>
+    <tr>
+      <td>/table</td>
+      <td style="text-align: left">/DATABASE.TABLEâproject name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/table_exd</td>
+      <td style="text-align: left">/DATABASE.TABLEâproject name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/execute</td>
+      <td style="text-align: left">/job id</td>
+      <td style="text-align: left">Â </td>
+    </tr>
+    <tr>
+      <td>/execute_out</td>
+      <td style="text-align: left">/job id-step index</td>
+      <td style="text-align: left">Â </td>
+    </tr>
+    <tr>
+      <td>/kafaka</td>
+      <td style="text-align: left">/DATABASE.TABLE</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/streaming</td>
+      <td style="text-align: left">/DATABASE.TABLE</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/user</td>
+      <td style="text-align: left">/user name</td>
+      <td style="text-align: left">Â </td>
+    </tr>
+  </tbody>
+</table>
+
+<h2 id="section-1">æ¥çåæ°æ®</h2>
+
+<p>Kylinä»¥äºè¿å¶åèçæ ¼å¼å°åæ°æ®åå¨å¨HBaseä¸ï¼å¦æä½ 
æ³è¦æ¥çä¸äºåæ°æ®ï¼å¯ä»¥è¿è¡ï¼</p>
+
+<div class="highlight"><pre><code class="language-groff" 
data-lang="groff">./bin/metastore.sh list 
/path/to/store/metadata</code></pre></div>
+
+<p>ååºåå¨å¨æå®è·¯å¾ä¸çææå®ä½åæ°æ®ãç¶åè¿è¡ï¼</p>
+
+<div class="highlight"><pre><code class="language-groff" 
data-lang="groff">./bin/metastore.sh cat 
/path/to/store/entity/metadata.</code></pre></div>
+
+<p>æ¥çæä¸ªå®ä½çåæ°æ®ã</p>
+
 <h2 id="metadata">ä½¿ç¨äºè¿å¶åæ¥å¤ä»½ metadata</h2>
 
 <p>ææ¶ä½ éè¦å° Kylin ç metadata store ä» hbase 
å¤ä»½å°ç£çæä»¶ç³»ç»ãå¨è¿ç§æåµä¸ï¼åè®¾ä½ å¨é¨ç½² Kylin ç 
hadoop å½ä»¤è¡ï¼ææ²çï¼éï¼ä½ å¯ä»¥å°KYLIN_HOMEå¹¶è¿è¡ï¼</p>
@@ -181,6 +279,12 @@ kylin.metadata.url=kylin_metadata@hbase<
 
 <p>æ¥å°ä½ çå
æ°æ®å¯¼åºå°æ¬å°ç®å½ï¼è¿ä¸ªç®å½å¨KYLIN_HOME/metadata_backpsä¸ï¼å®çå½åè§åä½¿ç¨äºå½åæ¶é´ä½ä¸ºåæ°ï¼KYLIN_HOME/meta_backups/meta_year_month_day_hour_minute_second
 ã</p>
 
+<p>æ¤å¤, ä½ å¯ä»¥è¿è¡:</p>
+
+<div class="highlight"><pre><code class="language-groff" 
data-lang="groff">./bin/metastore.sh fetch 
/path/to/store/metadata</code></pre></div>
+
+<p>æéæ©å°å¯¼åºåæ°æ®. ä¸¾ä¸ªæ å, è¿è¡ <code 
class="highlighter-rouge">./bin/metastore.sh fetch /cube_desc/</code> 
è·åææçcube descåæ°æ®, æèè¿è¡ <code 
class="highlighter-rouge">./bin/metastore.sh fetch 
/cube_desc/kylin_sales_cube.json</code> å¯¼åºåä¸ªcube descçåæ°æ®ã</p>
+
 <h2 id="metadata-1">ä½¿ç¨äºè¿å¶åæ¥æ¢å¤ metadata</h2>
 
 <p>ä¸ä¸ä½ åç°ä½ çåæ°æ®è¢«æå¾ä¸å¢ç³ï¼æ³è¦æ¢å¤å
åçå¤ä»½ï¼</p>
@@ -214,7 +318,7 @@ mkdir /path/to/restore_new/cube</code></
 
 <p>åªæå¨æ¤æä»¶å¤¹ä¸çæä»¶æä¼ä¸ä¼ å°Kylin Metastoreã åæ 
·ï¼å¨æ¢å¤å®æåï¼åå» Web UI ä¸çâReload 
Metadataâæé®ä»¥å·æ°ç¼åã</p>
 
-<h2 id="section">å¨å¼åç¯å¢å¤ä»½/æ¢å¤åæ°æ®</h2>
+<h2 id="section-2">å¨å¼åç¯å¢å¤ä»½/æ¢å¤åæ°æ®</h2>
 
 <p>å¨å¼åè°è¯ Kylin æ¶ï¼å¸åçç¯å¢æ¯ä¸å°è£æ IDE 
çå¼åæºä¸åä¸ä¸ªåå°çæ²çï¼éå¸¸ä½ ä¼åä»£ç 
å¹¶å¨å¼åæºä¸è¿è¡æµè¯æ¡ä¾ï¼ä½æ¯æ¬¡é½éè¦å°äºè¿å¶å
æ¾å°æ²çéä»¥æ£æ¥åæ°æ®æ¯å¾éº»ç¦çãè¿æ¶æä¸ä¸ªåä¸º 
SandboxMetastoreCLI å·¥å·ç±»å¯ä»¥å¸®å©ä½ å¨å¼åæºæ¬å°ä¸è½½/ä¸ä¼ å
æ°æ®ã</p>
 

Modified: kylin/site/docs/howto/howto_backup_metadata.html
URL: 
http://svn.apache.org/viewvc/kylin/site/docs/howto/howto_backup_metadata.html?rev=1856055&r1=1856054&r2=1856055&view=diff
==============================================================================
--- kylin/site/docs/howto/howto_backup_metadata.html (original)
+++ kylin/site/docs/howto/howto_backup_metadata.html Fri Mar 22 14:12:46 2019
@@ -6352,6 +6352,104 @@ kylin.metadata.url=kylin_metadata@hbase<
 
 <p>This indicates that the metadata will be saved as a htable called <code 
class="highlighter-rouge">kylin_metadata</code>. You can scan the htable in 
hbase shell to check it out.</p>
 
+<h2 id="metadata-directory">Metadata directory</h2>
+
+<p>Kylin metastore use <code class="highlighter-rouge">resource root path + 
resource name + resource suffix</code> as key (rowkey in hbase) to store 
metadata. You can refer to the following table to use <code 
class="highlighter-rouge">./bin/metastore.sh</code>.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th>Resource root path</th>
+      <th style="text-align: left">resource name</th>
+      <th style="text-align: left">resource suffix</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>/cube</td>
+      <td style="text-align: left">/cube name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/cube_desc</td>
+      <td style="text-align: left">/cube name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/cube_statistics</td>
+      <td style="text-align: left">/cube name/uuid</td>
+      <td style="text-align: left">.seq</td>
+    </tr>
+    <tr>
+      <td>/model_desc</td>
+      <td style="text-align: left">/model name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/dict</td>
+      <td style="text-align: left">/DATABASE.TABLE/COLUMN/uuid</td>
+      <td style="text-align: left">.dict</td>
+    </tr>
+    <tr>
+      <td>/project</td>
+      <td style="text-align: left">/project name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/table_snapshot</td>
+      <td style="text-align: left">/DATABASE.TABLE/uuid</td>
+      <td style="text-align: left">.snapshot</td>
+    </tr>
+    <tr>
+      <td>/table</td>
+      <td style="text-align: left">/DATABASE.TABLEâproject name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/table_exd</td>
+      <td style="text-align: left">/DATABASE.TABLEâproject name</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/execute</td>
+      <td style="text-align: left">/job id</td>
+      <td style="text-align: left">Â </td>
+    </tr>
+    <tr>
+      <td>/execute_out</td>
+      <td style="text-align: left">/job id-step index</td>
+      <td style="text-align: left">Â </td>
+    </tr>
+    <tr>
+      <td>/kafaka</td>
+      <td style="text-align: left">/DATABASE.TABLE</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/streaming</td>
+      <td style="text-align: left">/DATABASE.TABLE</td>
+      <td style="text-align: left">.json</td>
+    </tr>
+    <tr>
+      <td>/user</td>
+      <td style="text-align: left">/user name</td>
+      <td style="text-align: left">Â </td>
+    </tr>
+  </tbody>
+</table>
+
+<h2 id="view-metadata">View metadata</h2>
+
+<p>Kylin store metadata in Byte format in HBase. If you want to view some 
metadata, you can run:</p>
+
+<div class="highlight"><pre><code class="language-groff" 
data-lang="groff">./bin/metastore.sh list 
/path/to/store/metadata</code></pre></div>
+
+<p>to list the entity stored in specified directory, and then run:</p>
+
+<div class="highlight"><pre><code class="language-groff" 
data-lang="groff">./bin/metastore.sh cat 
/path/to/store/entity/metadata.</code></pre></div>
+
+<p>to view one entity metadata.</p>
+
 <h2 id="backup-metadata-with-binary-package">Backup metadata with binary 
package</h2>
 
 <p>Sometimes you need to backup the Kylinâs metadata store from hbase to 
your disk file system.<br />
@@ -6361,6 +6459,12 @@ In such cases, assuming youâre on t
 
 <p>to dump your metadata to your local folder a folder under 
KYLIN_HOME/metadata_backps, the folder is named after current time with the 
syntax: KYLIN_HOME/meta_backups/meta_year_month_day_hour_minute_second</p>
 
+<p>In addition, you can run:</p>
+
+<div class="highlight"><pre><code class="language-groff" 
data-lang="groff">./bin/metastore.sh fetch 
/path/to/store/metadata</code></pre></div>
+
+<p>to dump metadata selectively. For example, run <code 
class="highlighter-rouge">./bin/metastore.sh fetch /cube_desc/</code> to get 
all cube desc metadata, or run <code 
class="highlighter-rouge">./bin/metastore.sh fetch 
/cube_desc/kylin_sales_cube.json</code> to get single cube desc metadata.</p>
+
 <h2 id="restore-metadata-with-binary-package">Restore metadata with binary 
package</h2>
 
 <p>In case you find your metadata store messed up, and you want to restore to 
a previous backup:</p>

Modified: kylin/site/feed.xml
URL: 
http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1856055&r1=1856054&r2=1856055&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Fri Mar 22 14:12:46 2019
@@ -19,8 +19,8 @@
     <description>Apache Kylin Home</description>
     <link>http://kylin.apache.org/</link>
     <atom:link href="http://kylin.apache.org/feed.xml"; rel="self" 
type="application/rss+xml"/>
-    <pubDate>Mon, 18 Mar 2019 18:21:12 -0700</pubDate>
-    <lastBuildDate>Mon, 18 Mar 2019 18:21:12 -0700</lastBuildDate>
+    <pubDate>Fri, 22 Mar 2019 06:59:28 -0700</pubDate>
+    <lastBuildDate>Fri, 22 Mar 2019 06:59:28 -0700</lastBuildDate>
     <generator>Jekyll v2.5.3</generator>
     
       <item>
@@ -570,7 +570,7 @@ Graphic 4 Druid Schema&lt;/p&gt;
 
 &lt;p&gt;Kylin stores its data in HBase by converting the Dimensions and 
Measures into HBase Keys and Values, respectively. As HBase doesnât support 
secondary index and only has one RowKey index, Kylinâs Dimension values will 
be combined into a fixed sequence to store as RowKey. In this way, filtering on 
a Dimension in the front of the sequence will perform better than those at the 
back. Hereâs an example:&lt;/p&gt;
 
-&lt;p&gt;In the testing environment, there are two almost identical Cubes 
(Cube1 and Cube2). They both have the same data source and the same 
Dimensions/Measures. The only difference is the order of the Dimensions in the 
RowKey: Cube1 puts P_LINEORDER.LO_CUSTKEY at the first while Cube2 the 
last.&lt;br /&gt;
+&lt;p&gt;In the testing environment, there are two almost identical Cubes 
(Cube1 and Cube2). They both have the same data source and the same 
Dimensions/Measures. The only difference is the order of the Dimensions in the 
RowKey: Cube1 puts P_LINEORDER.LO_ORDERKEY at the first while Cube2 the 
last.&lt;br /&gt;
 &lt;img src=&quot;/images/blog/Kylin-On-Durid/5 
cube1_rowkey_sequence.png&quot; alt=&quot;&quot; /&gt;&lt;br /&gt;
 Graphic 5 Cube1 RowKey Sequence&lt;/p&gt;

svn commit: r1856055 - in /kylin/site: blog/2015/12/25/support-powerbi-tableau9/index.html blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/index.html cn/docs/howto/howto_backup_metadata.html docs/howto/howto_backup_metadata.html feed.xml

Reply via email to