Author: samarth
Date: Thu Mar 23 23:35:42 2017
New Revision: 1788343
URL: http://svn.apache.org/viewvc?rev=1788343&view=rev
Log:
Add page for column mapping and immutable data encoding
Added:
phoenix/site/publish/columnencoding.html
phoenix/site/source/src/site/markdown/columnencoding.md
Modified:
phoenix/site/publish/language/datatypes.html
phoenix/site/publish/language/functions.html
phoenix/site/publish/language/index.html
Added: phoenix/site/publish/columnencoding.html
URL:
http://svn.apache.org/viewvc/phoenix/site/publish/columnencoding.html?rev=1788343&view=auto
==============================================================================
--- phoenix/site/publish/columnencoding.html (added)
+++ phoenix/site/publish/columnencoding.html Thu Mar 23 23:35:42 2017
@@ -0,0 +1,527 @@
+
+<!DOCTYPE html>
+<!--
+ Generated by Apache Maven Doxia at 2017-03-23
+ Rendered using Reflow Maven Skin 1.1.0
(http://andriusvelykis.github.io/reflow-maven-skin)
+-->
+<html xml:lang="en" lang="en">
+
+ <head>
+ <meta charset="UTF-8" />
+ <title>Column Mapping and Immutable Data Encoding | Apache
Phoenix</title>
+ <meta name="viewport" content="width=device-width,
initial-scale=1.0" />
+ <meta name="description" content="" />
+ <meta http-equiv="content-language" content="en" />
+
+ <link
href="//netdna.bootstrapcdn.com/bootswatch/2.3.2/flatly/bootstrap.min.css"
rel="stylesheet" />
+ <link
href="//netdna.bootstrapcdn.com/twitter-bootstrap/2.3.1/css/bootstrap-responsive.min.css"
rel="stylesheet" />
+ <link href="./css/bootswatch.css" rel="stylesheet" />
+ <link href="./css/reflow-skin.css" rel="stylesheet" />
+
+ <link href="//yandex.st/highlightjs/7.5/styles/default.min.css"
rel="stylesheet" />
+
+ <link href="./css/lightbox.css" rel="stylesheet" />
+
+ <link href="./css/site.css" rel="stylesheet" />
+ <link href="./css/print.css" rel="stylesheet" media="print" />
+
+ <!-- Le HTML5 shim, for IE6-8 support of HTML5 elements -->
+ <!--[if lt IE 9]>
+ <script
src="http://html5shim.googlecode.com/svn/trunk/html5.js";></script>
+ <![endif]-->
+
+
+
+ </head>
+
+ <body class="page-columnencoding project-phoenix-site"
data-spy="scroll" data-offset="60" data-target="#toc-scroll-target">
+
+ <div class="navbar navbar-fixed-top">
+ <div class="navbar-inner">
+ <div class="container">
+ <a class="btn btn-navbar"
data-toggle="collapse" data-target="#top-nav-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+ <a class="brand" href="index.html"><div
class="xtoplogo"></div></a>
+ <div class="nav-collapse collapse"
id="top-nav-collapse">
+ <ul class="nav pull-right">
+ <li class="dropdown">
+ <a href="#"
class="dropdown-toggle" data-toggle="dropdown">About <b class="caret"></b></a>
+ <ul
class="dropdown-menu">
+ <li ><a
href="index.html" title="Overview">Overview</a></li>
+ <li ><a
href="who_is_using.html" title="Who is Using">Who is Using</a></li>
+ <li ><a
href="recent.html" title="New Features">New Features</a></li>
+ <li ><a
href="roadmap.html" title="Roadmap">Roadmap</a></li>
+ <li ><a
href="performance.html" title="Performance">Performance</a></li>
+ <li ><a
href="team.html" title="Team">Team</a></li>
+ <li ><a
href="resources.html" title="Presentations">Presentations</a></li>
+ <li ><a
href="mailing_list.html" title="Mailing Lists">Mailing Lists</a></li>
+ <li ><a
href="source.html" title="Source Repository">Source Repository</a></li>
+ <li ><a
href="issues.html" title="Issue Tracking">Issue Tracking</a></li>
+ <li ><a
href="download.html" title="Download">Download</a></li>
+ <li ><a
href="installation.html" title="Installation">Installation</a></li>
+ <li
class="divider"/>
+ <li ><a
href="contributing.html" title="How to Contribute">How to Contribute</a></li>
+ <li ><a
href="develop.html" title="How to Develop">How to Develop</a></li>
+ <li ><a
href="building_website.html" title="How to Update Website">How to Update
Website</a></li>
+ <li ><a
href="release.html" title="How to Release">How to Release</a></li>
+ <li
class="divider"/>
+ <li ><a
href="http://www.apache.org/licenses/"; title="License"
class="externalLink">License</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#"
class="dropdown-toggle" data-toggle="dropdown">Using <b class="caret"></b></a>
+ <ul
class="dropdown-menu">
+ <li ><a
href="faq.html" title="F.A.Q.">F.A.Q.</a></li>
+ <li ><a
href="Phoenix-in-15-minutes-or-less.html" title="Quick Start">Quick
Start</a></li>
+ <li ><a
href="building.html" title="Building">Building</a></li>
+ <li ><a
href="tuning.html" title="Tuning">Tuning</a></li>
+ <li ><a
href="upgrading.html" title="Backward Compatibility">Backward
Compatibility</a></li>
+ <li ><a
href="release_notes.html" title="Release Notes">Release Notes</a></li>
+ <li ><a
href="pherf.html" title="Performance Testing">Performance Testing</a></li>
+ <li
class="divider"/>
+ <li ><a
href="phoenix_spark.html" title="Apache Spark Integration">Apache Spark
Integration</a></li>
+ <li ><a
href="hive_storage_handler.html" title="Phoenix Storage Handler for Apache
Hive">Phoenix Storage Handler for Apache Hive</a></li>
+ <li ><a
href="pig_integration.html" title="Apache Pig Integration">Apache Pig
Integration</a></li>
+ <li ><a
href="phoenix_mr.html" title="Map Reduce Integration">Map Reduce
Integration</a></li>
+ <li ><a
href="flume.html" title="Apache Flume Plugin">Apache Flume Plugin</a></li>
+ <li ><a
href="kafka.html" title="Apache Kafka Plugin">Apache Kafka Plugin</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#"
class="dropdown-toggle" data-toggle="dropdown">Addons <b class="caret"></b></a>
+ <ul
class="dropdown-menu">
+ <li ><a
href="http://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-phoenix.html";
title="Phoenix on Amazon EMR" class="externalLink">Phoenix on Amazon
EMR</a></li>
+ <li ><a
href="http://python-phoenixdb.readthedocs.io/en/latest"; title="Phoenix Adapter
for Python" class="externalLink">Phoenix Adapter for Python</a></li>
+ <li ><a
href="phoenix_orm.html" title="Phoenix ORM Library">Phoenix ORM Library</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#"
class="dropdown-toggle" data-toggle="dropdown">Features <b
class="caret"></b></a>
+ <ul
class="dropdown-menu">
+ <li ><a
href="transactions.html" title="Transactions">Transactions</a></li>
+ <li ><a
href="udf.html" title="User-defined Functions">User-defined Functions</a></li>
+ <li
class="divider"/>
+ <li ><a
href="secondary_indexing.html" title="Secondary Indexes">Secondary
Indexes</a></li>
+ <li ><a
href="atomic_upsert.html" title="Atomic Upsert">Atomic Upsert</a></li>
+ <li ><a
href="namspace_mapping.html" title="Namespace Mapping">Namespace
Mapping</a></li>
+ <li ><a
href="update_statistics.html" title="Statistics Collection">Statistics
Collection</a></li>
+ <li ><a
href="rowtimestamp.html" title="Row Timestamp Column">Row Timestamp
Column</a></li>
+ <li ><a
href="paged.html" title="Paged Queries">Paged Queries</a></li>
+ <li ><a
href="salted.html" title="Salted Tables">Salted Tables</a></li>
+ <li ><a
href="skip_scan.html" title="Skip Scan">Skip Scan</a></li>
+ <li
class="divider"/>
+ <li ><a
href="views.html" title="Views">Views</a></li>
+ <li ><a
href="multi-tenancy.html" title="Multi tenancy">Multi tenancy</a></li>
+ <li ><a
href="dynamic_columns.html" title="Dynamic Columns">Dynamic Columns</a></li>
+ <li
class="divider"/>
+ <li ><a
href="bulk_dataload.html" title="Bulk Loading">Bulk Loading</a></li>
+ <li ><a
href="server.html" title="Query Server">Query Server</a></li>
+ <li ><a
href="tracing.html" title="Tracing">Tracing</a></li>
+ <li ><a
href="metrics.html" title="Metrics">Metrics</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#"
class="dropdown-toggle" data-toggle="dropdown">Reference <b
class="caret"></b></a>
+ <ul
class="dropdown-menu">
+ <li ><a
href="language/index.html" title="Grammar">Grammar</a></li>
+ <li ><a
href="language/functions.html" title="Functions">Functions</a></li>
+ <li ><a
href="language/datatypes.html" title="Datatypes">Datatypes</a></li>
+ <li ><a
href="array_type.html" title="ARRAY type">ARRAY type</a></li>
+ <li
class="divider"/>
+ <li ><a
href="sequences.html" title="Sequences">Sequences</a></li>
+ <li ><a
href="joins.html" title="Joins">Joins</a></li>
+ <li ><a
href="subqueries.html" title="Subqueries">Subqueries</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </div>
+ </div>
+
+ <div class="container">
+
+ <!-- Masthead
+ ================================================== -->
+
+ <header>
+ </header>
+
+ <div class="main-body">
+ <div class="row">
+ <div class="span12">
+ <div class="body-content">
+<div class="page-header">
+ <h1>Column Mapping and Immutable Data Encoding</h1>
+</div>
+<p>As part of Phoenix 4.10, we have released a new feature that provides a
layer of indirection between phoenix column names and the corresponding hbase
column qualifiers. We have also introduced a new encoding scheme for immutable
tables. For more details on benefits of using column mapping and immutable data
encoding, go to this <a class="externalLink"
href="https://blogs.apache.org/phoenix/entry/column-mapping-and-immutable-data";>blog</a>.</p>
+<div class="section">
+ <div class="section">
+ <h3 id="How_to_use_column_mapping">How to use column mapping?</h3>
+ <p>One can set the column mapping property only at the time of creating the
table. Before deciding on using column mapping, you need to think about how
many columns you expect in a table and its view hierarchy to have in the
lifecycle. For various mapping schemes, below are the limits on number of
columns:</p>
+ <table border="0" class="bodyTable table table-striped table-hover">
+ <thead>
+ <tr class="a">
+ <th>Config/Property Value </th>
+ <th>Max # of columns</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr class="b">
+ <td>1 </td>
+ <td>255</td>
+ </tr>
+ <tr class="a">
+ <td>2 </td>
+ <td>65535</td>
+ </tr>
+ <tr class="b">
+ <td>3 </td>
+ <td>16777215</td>
+ </tr>
+ <tr class="a">
+ <td>4 </td>
+ <td>2147483647</td>
+ </tr>
+ <tr class="b">
+ <td>NONE </td>
+ <td>no limit(theoretically)</td>
+ </tr>
+ </tbody>
+ </table>
+ <p>For mutable tables, this limit applies to columns in <b>all</b> column
families. For immutable tables, the limit applies to <b>per</b> column family.
By default, any new phoenix tables will be using the column mapping feature.
These defaults could be overridden by setting below config to the desired value
in hbase-site.xml.</p>
+ <table border="0" class="bodyTable table table-striped table-hover">
+ <thead>
+ <tr class="a">
+ <th>Table type </th>
+ <th>Default Column mapping </th>
+ <th>Config</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr class="b">
+ <td>Mutable/Immutable </td>
+ <td>2 byte qualifiers </td>
+ <td>phoenix.default.column.encoded.bytes.attrib</td>
+ </tr>
+ </tbody>
+ </table>
+ <p>Keep in mind that this config controls global level defaults that would
apply to all tables. If you would like to use a different mapping scheme than
this global default, then you can use the COLUMN_ENCODED_BYTES table
property.</p>
+ <div class="source">
+ <pre>CREATE TABLE T
+(
+a_string varchar not null,
+col1 integer
+CONSTRAINT pk PRIMARY KEY (a_string)
+)
+COLUMN_ENCODED_BYTES = 1;
+</pre>
+ </div>
+ </div>
+ <div class="section">
+ <h3 id="How_to_use_immutable_data_encoding">How to use immutable data
encoding?</h3>
+ <p>Like column mapping, one can set the immutable data encoding only at the
time of creating the table. Through our performance testing, we have found that
using SINGLE_CELL_ARRAY_WITH_OFFSETS generally provides really good performance
improvement and space savings. Below are some scenarios in when it would be
better to use ONE_CELL_PER_COLUMN encoding instead.</p>
+ <ul>
+ <li>Data is sparse i.e. less than 50% of the columns have values</li>
+ <li>Size of data within a column family gets too big. Our general guidance
here is that with default HBase block size of 64K, if data within a column
family grows beyond 50K then we shouldnât be using
SINGLE_CELL_ARRAY_WITH_OFFSETS.</li>
+ <li>For immutable tables that are going to have views on them</li>
+ </ul>
+ <p>By default, immutable non-multitenant tables are created using the two
byte column mapping and the SINGLE_CELL_ARRAY_WITH_OFFSETS data encoding. On
the other hand, immutable multi-tenant tables are created with two byte column
mapping scheme and ONE_COLUMN_PER_CELL data encoding. This is because we
generally expect users to create tenant specific views on the base multi-tenant
tables which as mentioned in the criteria above is more suitable for
ONE_CELL_PER_COLUMN encoding. Like column mapping, if you would like to change
these global defaults, you can do so by setting below configs in the
hbase-site.xml.</p>
+ <table border="0" class="bodyTable table table-striped table-hover">
+ <thead>
+ <tr class="a">
+ <th>Immutable Table type </th>
+ <th>Immutable storage scheme </th>
+ <th>Config</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr class="b">
+ <td>Multi-tenant </td>
+ <td>ONE_CELL_PER_COLUMN </td>
+ <td>phoenix.default.multitenant.immutable.storage.scheme</td>
+ </tr>
+ <tr class="a">
+ <td>Non multi-tenant </td>
+ <td>SINGLE_CELL_ARRAY_WITH_OFFSETS </td>
+ <td>phoenix.default.immutable.storage.scheme</td>
+ </tr>
+ </tbody>
+ </table>
+ <p>One could also provide specific immutable storage scheme and column
mapping scheme using the table properties IMMUTABLE_STORAGE_SCHEME and
COLUMN_ENCODED_BYTES. For example - </p>
+ <div class="source">
+ <pre>CREATE IMMUTABLE TABLE T
+(
+a_string varchar not null,
+col1 integer
+CONSTRAINT pk PRIMARY KEY (a_string)
+)
+IMMUTABLE_STORAGE_SCHEME = SINGLE_CELL_ARRAY_WITH_OFFSETS,
+COLUMN_ENCODED_BYTES = 1;
+</pre>
+ </div>
+ <p>One could chose to not use the SINGLE_CELL_ARRAY_WITH_OFFSETS encoding
but still use one of the number based column mapping. Here is an example - </p>
+ <div class="source">
+ <pre>CREATE IMMUTABLE TABLE T
+(
+a_string varchar not null,
+col1 integer
+CONSTRAINT pk PRIMARY KEY (a_string)
+)
+IMMUTABLE_STORAGE_SCHEME = ONE_CELL_PER_COLUMN,
+COLUMN_ENCODED_BYTES = 1;
+</pre>
+ </div>
+ <p>When using SINGLE_CELL_ARRAY_WITH_OFFSETS encoding, one has to use a
number based column mapping scheme. An attempt to use
SINGLE_CELL_ARRAY_WITH_OFFSETS with COLUMN_ENCODED_BYTES = NONE will throw an
error.</p>
+ </div>
+</div>
+ </div>
+ </div>
+ </div>
+ </div>
+
+ </div><!-- /container -->
+
+ <!-- Footer
+ ================================================== -->
+ <footer class="well">
+ <div class="container">
+ <div class="row">
+ <div class="span2 bottom-nav">
+ <ul class="nav nav-list">
+ <li
class="nav-header">About</li>
+ <li >
+ <a href="index.html"
title="Overview">Overview</a>
+ </li>
+ <li >
+ <a
href="who_is_using.html" title="Who is Using">Who is Using</a>
+ </li>
+ <li >
+ <a href="recent.html"
title="New Features">New Features</a>
+ </li>
+ <li >
+ <a href="roadmap.html"
title="Roadmap">Roadmap</a>
+ </li>
+ <li >
+ <a
href="performance.html" title="Performance">Performance</a>
+ </li>
+ <li >
+ <a href="team.html"
title="Team">Team</a>
+ </li>
+ <li >
+ <a
href="resources.html" title="Presentations">Presentations</a>
+ </li>
+ <li >
+ <a
href="mailing_list.html" title="Mailing Lists">Mailing Lists</a>
+ </li>
+ <li >
+ <a href="source.html"
title="Source Repository">Source Repository</a>
+ </li>
+ <li >
+ <a href="issues.html"
title="Issue Tracking">Issue Tracking</a>
+ </li>
+ <li >
+ <a href="download.html"
title="Download">Download</a>
+ </li>
+ <li >
+ <a
href="installation.html" title="Installation">Installation</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a
href="contributing.html" title="How to Contribute">How to Contribute</a>
+ </li>
+ <li >
+ <a href="develop.html"
title="How to Develop">How to Develop</a>
+ </li>
+ <li >
+ <a
href="building_website.html" title="How to Update Website">How to Update
Website</a>
+ </li>
+ <li >
+ <a href="release.html"
title="How to Release">How to Release</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a
href="http://www.apache.org/licenses/"; title="License"
class="externalLink">License</a>
+ </li>
+ </ul>
+ </div>
+ <div class="span2 bottom-nav">
+ <ul class="nav nav-list">
+ <li
class="nav-header">Using</li>
+ <li >
+ <a href="faq.html"
title="F.A.Q.">F.A.Q.</a>
+ </li>
+ <li >
+ <a
href="Phoenix-in-15-minutes-or-less.html" title="Quick Start">Quick Start</a>
+ </li>
+ <li >
+ <a href="building.html"
title="Building">Building</a>
+ </li>
+ <li >
+ <a href="tuning.html"
title="Tuning">Tuning</a>
+ </li>
+ <li >
+ <a
href="upgrading.html" title="Backward Compatibility">Backward Compatibility</a>
+ </li>
+ <li >
+ <a
href="release_notes.html" title="Release Notes">Release Notes</a>
+ </li>
+ <li >
+ <a href="pherf.html"
title="Performance Testing">Performance Testing</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a
href="phoenix_spark.html" title="Apache Spark Integration">Apache Spark
Integration</a>
+ </li>
+ <li >
+ <a
href="hive_storage_handler.html" title="Phoenix Storage Handler for Apache
Hive">Phoenix Storage Handler for Apache Hive</a>
+ </li>
+ <li >
+ <a
href="pig_integration.html" title="Apache Pig Integration">Apache Pig
Integration</a>
+ </li>
+ <li >
+ <a
href="phoenix_mr.html" title="Map Reduce Integration">Map Reduce Integration</a>
+ </li>
+ <li >
+ <a href="flume.html"
title="Apache Flume Plugin">Apache Flume Plugin</a>
+ </li>
+ <li >
+ <a href="kafka.html"
title="Apache Kafka Plugin">Apache Kafka Plugin</a>
+ </li>
+ </ul>
+ </div>
+ <div class="span2 bottom-nav">
+ <ul class="nav nav-list">
+ <li
class="nav-header">Features</li>
+ <li >
+ <a
href="transactions.html" title="Transactions">Transactions</a>
+ </li>
+ <li >
+ <a href="udf.html"
title="User-defined Functions">User-defined Functions</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a
href="secondary_indexing.html" title="Secondary Indexes">Secondary Indexes</a>
+ </li>
+ <li >
+ <a
href="atomic_upsert.html" title="Atomic Upsert">Atomic Upsert</a>
+ </li>
+ <li >
+ <a
href="namspace_mapping.html" title="Namespace Mapping">Namespace Mapping</a>
+ </li>
+ <li >
+ <a
href="update_statistics.html" title="Statistics Collection">Statistics
Collection</a>
+ </li>
+ <li >
+ <a
href="rowtimestamp.html" title="Row Timestamp Column">Row Timestamp Column</a>
+ </li>
+ <li >
+ <a href="paged.html"
title="Paged Queries">Paged Queries</a>
+ </li>
+ <li >
+ <a href="salted.html"
title="Salted Tables">Salted Tables</a>
+ </li>
+ <li >
+ <a
href="skip_scan.html" title="Skip Scan">Skip Scan</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a href="views.html"
title="Views">Views</a>
+ </li>
+ <li >
+ <a
href="multi-tenancy.html" title="Multi tenancy">Multi tenancy</a>
+ </li>
+ <li >
+ <a
href="dynamic_columns.html" title="Dynamic Columns">Dynamic Columns</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a
href="bulk_dataload.html" title="Bulk Loading">Bulk Loading</a>
+ </li>
+ <li >
+ <a href="server.html"
title="Query Server">Query Server</a>
+ </li>
+ <li >
+ <a href="tracing.html"
title="Tracing">Tracing</a>
+ </li>
+ <li >
+ <a href="metrics.html"
title="Metrics">Metrics</a>
+ </li>
+ </ul>
+ </div>
+ <div class="span3 bottom-nav">
+ <ul class="nav nav-list">
+ <li
class="nav-header">Reference</li>
+ <li >
+ <a
href="language/index.html" title="Grammar">Grammar</a>
+ </li>
+ <li >
+ <a
href="language/functions.html" title="Functions">Functions</a>
+ </li>
+ <li >
+ <a
href="language/datatypes.html" title="Datatypes">Datatypes</a>
+ </li>
+ <li >
+ <a
href="array_type.html" title="ARRAY type">ARRAY type</a>
+ </li>
+ <li >
+ <a href="http:divider"
title=""></a>
+ </li>
+ <li >
+ <a
href="sequences.html" title="Sequences">Sequences</a>
+ </li>
+ <li >
+ <a href="joins.html"
title="Joins">Joins</a>
+ </li>
+ <li >
+ <a
href="subqueries.html" title="Subqueries">Subqueries</a>
+ </li>
+ </ul>
+ </div>
+ <div class="span3 bottom-description">
+ <form
action="http://search-hadoop.com/?"; method="get"><input value="Phoenix"
name="fc_project" type="hidden"><input placeholder="Search Phoenix…"
required="required" style="width:170px;" size="18" name="q" id="query"
type="search"></form>
+ </div>
+ </div>
+ </div>
+ </footer>
+
+ <div class="container subfooter">
+ <div class="row">
+ <div class="span12">
+ <p class="pull-right"><a href="#">Back to
top</a></p>
+ <p class="copyright">Copyright ©2017 <a
href="http://www.apache.org";>Apache Software Foundation</a>. All Rights
Reserved.</p>
+ </div>
+ </div>
+ </div>
+
+ <!-- Le javascript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script
src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
+
+ <script
src="//netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/js/bootstrap.min.js"></script>
+ <script src="./js/lightbox.js"></script>
+ <script src="./js/jquery.smooth-scroll.min.js"></script>
+ <!-- back button support for smooth scroll -->
+ <script src="./js/jquery.ba-bbq.min.js"></script>
+ <script src="//yandex.st/highlightjs/7.5/highlight.min.js"></script>
+
+ <script src="./js/reflow-skin.js"></script>
+
+ </body>
+</html>
Modified: phoenix/site/publish/language/datatypes.html
URL:
http://svn.apache.org/viewvc/phoenix/site/publish/language/datatypes.html?rev=1788343&r1=1788342&r2=1788343&view=diff
==============================================================================
--- phoenix/site/publish/language/datatypes.html (original)
+++ phoenix/site/publish/language/datatypes.html Thu Mar 23 23:35:42 2017
@@ -1,7 +1,7 @@
<!DOCTYPE html>
<!--
- Generated by Apache Maven Doxia at 2017-03-20
+ Generated by Apache Maven Doxia at 2017-03-23
Rendered using Reflow Maven Skin 1.1.0
(http://andriusvelykis.github.io/reflow-maven-skin)
-->
<html xml:lang="en" lang="en">
Modified: phoenix/site/publish/language/functions.html
URL:
http://svn.apache.org/viewvc/phoenix/site/publish/language/functions.html?rev=1788343&r1=1788342&r2=1788343&view=diff
==============================================================================
--- phoenix/site/publish/language/functions.html (original)
+++ phoenix/site/publish/language/functions.html Thu Mar 23 23:35:42 2017
@@ -1,7 +1,7 @@
<!DOCTYPE html>
<!--
- Generated by Apache Maven Doxia at 2017-03-20
+ Generated by Apache Maven Doxia at 2017-03-23
Rendered using Reflow Maven Skin 1.1.0
(http://andriusvelykis.github.io/reflow-maven-skin)
-->
<html xml:lang="en" lang="en">
Modified: phoenix/site/publish/language/index.html
URL:
http://svn.apache.org/viewvc/phoenix/site/publish/language/index.html?rev=1788343&r1=1788342&r2=1788343&view=diff
==============================================================================
--- phoenix/site/publish/language/index.html (original)
+++ phoenix/site/publish/language/index.html Thu Mar 23 23:35:42 2017
@@ -1,7 +1,7 @@
<!DOCTYPE html>
<!--
- Generated by Apache Maven Doxia at 2017-03-20
+ Generated by Apache Maven Doxia at 2017-03-23
Rendered using Reflow Maven Skin 1.1.0
(http://andriusvelykis.github.io/reflow-maven-skin)
-->
<html xml:lang="en" lang="en">
Added: phoenix/site/source/src/site/markdown/columnencoding.md
URL:
http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/columnencoding.md?rev=1788343&view=auto
==============================================================================
--- phoenix/site/source/src/site/markdown/columnencoding.md (added)
+++ phoenix/site/source/src/site/markdown/columnencoding.md Thu Mar 23 23:35:42
2017
@@ -0,0 +1,79 @@
+#Column Mapping and Immutable Data Encoding
+
+As part of Phoenix 4.10, we have released a new feature that provides a layer
of indirection between phoenix column names and the corresponding hbase column
qualifiers. We have also introduced a new encoding scheme for immutable tables.
For more details on benefits of using column mapping and immutable data
encoding, go to this
[blog](https://blogs.apache.org/phoenix/entry/column-mapping-and-immutable-data).
+
+
+###How to use column mapping?
+One can set the column mapping property only at the time of creating the
table. Before deciding on using column mapping, you need to think about how
many columns you expect in a table and its view hierarchy to have in the
lifecycle. For various mapping schemes, below are the limits on number of
columns:
+
+Config/Property Value | Max # of columns
+------------ | ------------- | -------------
+1 | 255
+2 | 65535
+3 | 16777215
+4 | 2147483647
+NONE | no limit(theoretically)
+
+For mutable tables, this limit applies to columns in **all** column families.
For immutable tables, the limit applies to **per** column family. By default,
any new phoenix tables will be using the column mapping feature. These defaults
could be overridden by setting below config to the desired value in
hbase-site.xml.
+
+Table type | Default Column mapping | Config
+------------ | ------------- | -------------
+Mutable/Immutable | 2 byte qualifiers |
phoenix.default.column.encoded.bytes.attrib
+
+Keep in mind that this config controls global level defaults that would apply
to all tables. If you would like to use a different mapping scheme than this
global default, then you can use the COLUMN_ENCODED_BYTES table property.
+
+```
+CREATE TABLE T
+(
+a_string varchar not null,
+col1 integer
+CONSTRAINT pk PRIMARY KEY (a_string)
+)
+COLUMN_ENCODED_BYTES = 1;
+```
+
+###How to use immutable data encoding?
+Like column mapping, one can set the immutable data encoding only at the time
of creating the table. Through our performance testing, we have found that
using SINGLE_CELL_ARRAY_WITH_OFFSETS generally provides really good performance
improvement and space savings. Below are some scenarios in when it would be
better to use ONE_CELL_PER_COLUMN encoding instead.
+
+- Data is sparse i.e. less than 50% of the columns have values
+- Size of data within a column family gets too big. Our general guidance here
is that with default HBase block size of 64K, if data within a column family
grows beyond 50K then we shouldnât be using SINGLE_CELL_ARRAY_WITH_OFFSETS.
+- For immutable tables that are going to have views on them
+
+By default, immutable non-multitenant tables are created using the two byte
column mapping and the SINGLE_CELL_ARRAY_WITH_OFFSETS data encoding. On the
other hand, immutable multi-tenant tables are created with two byte column
mapping scheme and ONE_COLUMN_PER_CELL data encoding. This is because we
generally expect users to create tenant specific views on the base multi-tenant
tables which as mentioned in the criteria above is more suitable for
ONE_CELL_PER_COLUMN encoding. Like column mapping, if you would like to change
these global defaults, you can do so by setting below configs in the
hbase-site.xml.
+
+Immutable Table type | Immutable storage scheme | Config
+------------ | ------------- | -------------
+Multi-tenant | ONE_CELL_PER_COLUMN |
phoenix.default.multitenant.immutable.storage.scheme
+Non multi-tenant | SINGLE_CELL_ARRAY_WITH_OFFSETS |
phoenix.default.immutable.storage.scheme
+
+One could also provide specific immutable storage scheme and column mapping
scheme using the table properties IMMUTABLE_STORAGE_SCHEME and
COLUMN_ENCODED_BYTES. For example -
+
+```
+CREATE IMMUTABLE TABLE T
+(
+a_string varchar not null,
+col1 integer
+CONSTRAINT pk PRIMARY KEY (a_string)
+)
+IMMUTABLE_STORAGE_SCHEME = SINGLE_CELL_ARRAY_WITH_OFFSETS,
+COLUMN_ENCODED_BYTES = 1;
+```
+
+One could chose to not use the SINGLE_CELL_ARRAY_WITH_OFFSETS encoding but
still use one of the number based column mapping. Here is an example -
+
+```
+CREATE IMMUTABLE TABLE T
+(
+a_string varchar not null,
+col1 integer
+CONSTRAINT pk PRIMARY KEY (a_string)
+)
+IMMUTABLE_STORAGE_SCHEME = ONE_CELL_PER_COLUMN,
+COLUMN_ENCODED_BYTES = 1;
+```
+
+When using SINGLE_CELL_ARRAY_WITH_OFFSETS encoding, one has to use a number
based column mapping scheme. An attempt to use SINGLE_CELL_ARRAY_WITH_OFFSETS
with COLUMN_ENCODED_BYTES = NONE will throw an error.
+
+
+
+
