Author: bridgetb
Date: Tue Mar 17 21:08:21 2015
New Revision: 1667403
URL: http://svn.apache.org/r1667403
Log:
merging gh-pages-master to gh-pages
Added:
drill/site/trunk/content/drill/docs/connect-to-a-data-source/
drill/site/trunk/content/drill/docs/connect-to-a-data-source/index.html
drill/site/trunk/content/drill/docs/data-sources-and-file-formats/
drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html
drill/site/trunk/content/drill/docs/data-type-casting/
drill/site/trunk/content/drill/docs/data-type-casting/index.html
drill/site/trunk/content/drill/docs/date-time-formats/
drill/site/trunk/content/drill/docs/date-time-formats/index.html
drill/site/trunk/content/drill/docs/deploying-and-using-a-hive-udf/
drill/site/trunk/content/drill/docs/deploying-and-using-a-hive-udf/index.html
drill/site/trunk/content/drill/docs/file-system-storage-plugin/
drill/site/trunk/content/drill/docs/file-system-storage-plugin/index.html
drill/site/trunk/content/drill/docs/getting-started-tutorial/
drill/site/trunk/content/drill/docs/getting-started-tutorial/index.html
drill/site/trunk/content/drill/docs/handling-disparate-data-types/
drill/site/trunk/content/drill/docs/handling-disparate-data-types/index.html
drill/site/trunk/content/drill/docs/hbase-storage-plugin/
drill/site/trunk/content/drill/docs/hbase-storage-plugin/index.html
drill/site/trunk/content/drill/docs/hive-storage-plugin/
drill/site/trunk/content/drill/docs/hive-storage-plugin/index.html
drill/site/trunk/content/drill/docs/hive-to-drill-data-type-mapping/
drill/site/trunk/content/drill/docs/hive-to-drill-data-type-mapping/index.html
drill/site/trunk/content/drill/docs/img/connect-plugin.png (with props)
drill/site/trunk/content/drill/docs/img/data-sources-schemachg.png (with
props)
drill/site/trunk/content/drill/docs/img/datasources-json-bracket.png
(with props)
drill/site/trunk/content/drill/docs/img/datasources-json.png (with props)
drill/site/trunk/content/drill/docs/img/get2kno_plugin.png (with props)
drill/site/trunk/content/drill/docs/img/json-workaround.png (with props)
drill/site/trunk/content/drill/docs/img/plugin-default.png (with props)
drill/site/trunk/content/drill/docs/json-data-model/
drill/site/trunk/content/drill/docs/json-data-model/index.html
drill/site/trunk/content/drill/docs/lesson-1-connect-to-data-sources/
drill/site/trunk/content/drill/docs/lesson-1-connect-to-data-sources/index.html
drill/site/trunk/content/drill/docs/lesson-2-query-plain-text/
drill/site/trunk/content/drill/docs/lesson-2-query-plain-text/index.html
drill/site/trunk/content/drill/docs/lesson-3-create-a-storage-plugin/
drill/site/trunk/content/drill/docs/lesson-3-create-a-storage-plugin/index.html
drill/site/trunk/content/drill/docs/lexical-structure/
drill/site/trunk/content/drill/docs/lexical-structure/index.html
drill/site/trunk/content/drill/docs/mapr-db-format/
drill/site/trunk/content/drill/docs/mapr-db-format/index.html
drill/site/trunk/content/drill/docs/parquet-format/
drill/site/trunk/content/drill/docs/parquet-format/index.html
drill/site/trunk/content/drill/docs/storage-plugin-configuration/
drill/site/trunk/content/drill/docs/storage-plugin-configuration/index.html
Modified:
drill/site/trunk/content/drill/docs/apache-drill-contribution-guidelines/index.html
drill/site/trunk/content/drill/docs/apache-drill-in-10-minutes/index.html
drill/site/trunk/content/drill/docs/drill-default-input-format/index.html
drill/site/trunk/content/drill/docs/getting-to-know-the-drill-sandbox/index.html
drill/site/trunk/content/drill/docs/img/Hbase_Browse.png
drill/site/trunk/content/drill/docs/index.html
drill/site/trunk/content/drill/docs/installing-the-mapr-sandbox-with-apache-drill-on-virtualbox/index.html
drill/site/trunk/content/drill/docs/installing-the-mapr-sandbox-with-apache-drill-on-vmware-player-vmware-fusion/index.html
drill/site/trunk/content/drill/docs/lession-1-learn-about-the-data-set/index.html
drill/site/trunk/content/drill/docs/lession-3-run-queries-on-complex-data-types/index.html
drill/site/trunk/content/drill/docs/persistent-configuration-storage/index.html
drill/site/trunk/content/drill/docs/planning-and-execution-options/index.html
drill/site/trunk/content/drill/docs/ports-used-by-drill/index.html
drill/site/trunk/content/drill/docs/querying-complex-data/index.html
drill/site/trunk/content/drill/docs/querying-hive/index.html
drill/site/trunk/content/drill/docs/reserved-keywords/index.html
drill/site/trunk/content/drill/docs/sql-commands-summary/index.html
drill/site/trunk/content/drill/docs/sql-functions/index.html
drill/site/trunk/content/drill/docs/sql-reference/index.html
drill/site/trunk/content/drill/docs/start-up-options/index.html
drill/site/trunk/content/drill/docs/starting-stopping-drill/index.html
drill/site/trunk/content/drill/docs/step-3-connect-to-drill-data-sources-from-a-bi-tool/index.html
drill/site/trunk/content/drill/docs/storage-plugin-registration/index.html
drill/site/trunk/content/drill/docs/tableau-examples/index.html
drill/site/trunk/content/drill/docs/workspaces/index.html
drill/site/trunk/content/drill/feed.xml
Modified:
drill/site/trunk/content/drill/docs/apache-drill-contribution-guidelines/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/apache-drill-contribution-guidelines/index.html?rev=1667403&r1=1667402&r2=1667403&view=diff
==============================================================================
---
drill/site/trunk/content/drill/docs/apache-drill-contribution-guidelines/index.html
(original)
+++
drill/site/trunk/content/drill/docs/apache-drill-contribution-guidelines/index.html
Tue Mar 17 21:08:21 2015
@@ -144,7 +144,7 @@ understand your requirements.</p>
<p>Setting up IDE formatters is recommended and can be done by importing the
following settings into your browser:</p>
-<p>IntelliJ IDEA formatter: <a
href="/confluence/download/attachments/30757399/idea-%0Asettings.jar?version=1&modificationDate=1363022308000&api=v2">settings
+<p>IntelliJ IDEA formatter: <a
href="https://cwiki.apache.org/confluence/download/attachments/30757399/idea-settings.jar?version=1&modificationDate=1363022308000&api=v2";>settings
jar</a></p>
<p>Eclipse: <a
href="https://issues.apache.org/jira/secure/atta%0Achment/12474245/eclipse_formatter_apache.xml";>formatter
xml from HBase</a></p>
Modified:
drill/site/trunk/content/drill/docs/apache-drill-in-10-minutes/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/apache-drill-in-10-minutes/index.html?rev=1667403&r1=1667402&r2=1667403&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/apache-drill-in-10-minutes/index.html
(original)
+++ drill/site/trunk/content/drill/docs/apache-drill-in-10-minutes/index.html
Tue Mar 17 21:08:21 2015
@@ -439,7 +439,7 @@ following tasks:</p>
<ul>
<li><a href="/docs/deploying-apache-drill-in-a-clustered-environment">Deploy
Drill in a clustered environment.</a></li>
<li><a href="/docs/connect-to-data-sources">Configure storage plugins to
connect Drill to your data sources</a>.</li>
-<li>Query <a href="/docs/querying-hive">Hive</a> and <a
href="/docs/registering-hbase">HBase</a> data.</li>
+<li>Query <a href="/docs/querying-hive">Hive</a> and <a
href="/docs/hbase-storage-plugin">HBase</a> data.</li>
<li><a href="/docs/querying-complex-data">Query Complex Data</a></li>
<li><a href="/docs/querying-plain-text-files">Query Plain Text Files</a></li>
</ul>
Added: drill/site/trunk/content/drill/docs/connect-to-a-data-source/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/connect-to-a-data-source/index.html?rev=1667403&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/connect-to-a-data-source/index.html
(added)
+++ drill/site/trunk/content/drill/docs/connect-to-a-data-source/index.html Tue
Mar 17 21:08:21 2015
@@ -0,0 +1,124 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Connect to a Data Source - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Connect to a Data Source</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>A storage plugin is an interface for
connecting to a data source to read and write data. Apache Drill connects to a
data source, such as a file on the file system or a Hive metastore, through a
storage plugin. When you execute a query, Drill gets the plugin name you
provide in FROM clause of your query or from the default you specify in the
USE.<plugin name> command that precedes the query.
+. </p>
+
+<p>In addition to the connection string, the storage plugin configures the
workspace and file formats for reading data, as described in subsequent
sections. </p>
+
+<h2 id="storage-plugins-internals">Storage Plugins Internals</h2>
+
+<p>The following image represents the storage plugin layer between Drill and a
+data source:</p>
+
+<p><img src="/docs/img/storageplugin.png" alt="drill query flow"></p>
+
+<p>A storage plugin provides the following information to Drill:</p>
+
+<ul>
+<li>Metadata available in the underlying data source</li>
+<li>Location of data</li>
+<li>Interfaces that Drill can use to read from and write to data sources</li>
+<li>A set of storage plugin optimization rules that assist with efficient and
faster execution of Drill queries, such as pushdowns, statistics, and partition
awareness</li>
+</ul>
+
+<p>A storage plugin performs scanner and writer functions, and informs the
metadata repository of any known metadata. The metadata repository is a
database created to store metadata. The metadata is information about the
structures that contain the actual data, such as:</p>
+
+<ul>
+<li>Schema</li>
+<li>File size</li>
+<li>Data ordering</li>
+<li>Secondary indices</li>
+<li>Number of blocks</li>
+</ul>
+
+<p>A storage plugin informs the execution engine of any native capabilities,
such
+as predicate pushdown, joins, and SQL.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Added:
drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html?rev=1667403&view=auto
==============================================================================
---
drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html
(added)
+++
drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html
Tue Mar 17 21:08:21 2015
@@ -0,0 +1,113 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Data Sources and File Formats - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Data Sources and File Formats</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>Included in the data sources that Drill
supports are these key data sources:</p>
+
+<ul>
+<li>Hbase</li>
+<li>Hive</li>
+<li>MapR-DB</li>
+<li>File system</li>
+</ul>
+
+<p>Drill supports the following input formats for data:</p>
+
+<ul>
+<li>CSV (Comma-Separated-Values)</li>
+<li>TSV (Tab-Separated-Values)</li>
+<li>PSV (Pipe-Separated-Values)</li>
+<li>Parquet</li>
+<li>JSON</li>
+</ul>
+
+<p>You set the input format for data coming from data sources to Drill in the
workspace portion of the <a href="/docs/storage-plugin-registration">storage
plugin</a> definition. The default input format in Drill is Parquet. </p>
+
+<p>You change the <a href="/docs/planning-and-execution-options">sys.options
table</a> to set the output format of Drill data. The default storage format
for Drill Create Table AS (CTAS) statements is Parquet.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Added: drill/site/trunk/content/drill/docs/data-type-casting/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/data-type-casting/index.html?rev=1667403&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/data-type-casting/index.html (added)
+++ drill/site/trunk/content/drill/docs/data-type-casting/index.html Tue Mar 17
21:08:21 2015
@@ -0,0 +1,238 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Data Type Casting - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Data Type Casting</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>The following table lists the type
arguments you can use with the cast function:</p>
+
+<table>
+ <tr>
+ <th>SQL Type</th>
+ <th>Drill Description</th>
+ </tr>
+ <tr>
+ <td>BIGINT</td>
+ <td>8-byte signed integer</td>
+ </tr>
+ <tr>
+ <td>BOOLEAN</td>
+ <td>True or false</td>
+ </tr>
+ <tr>
+ <td>DATE</td>
+ <td>Years, months, and days in YYYY-ÂMM-ÂDD format</td>
+ </tr>
+ <tr>
+ <td>NUMERIC, DECIMAL, or DEC(p,s)</td>
+ <td>38-digit precision number. Same as numeric(p,s) where precision is p,
and scale is s. Example: decimal(6,2) has 4 digits before the decimal point and
2 digits after the decimal point.</td>
+ </tr>
+ <tr>
+ <td>FLOAT</td>
+ <td>4-byte single precision floating point number</td>
+ </tr>
+ <tr>
+ <td>DOUBLE, DOUBLE PRECISION</td>
+ <td>8-byte double precision floating point number. </td>
+ </tr>
+ <tr>
+ <td>INTEGER, INT</td>
+ <td>4-byte signed integer</td>
+ </tr>
+ <tr>
+ <td>INTERVAL</td>
+ <td>Integer fields representing a period of time in years, months, day,s
hours, minutes, seconds and optional milliseconds using ISO 8601 format.</td>
+ </tr>
+ <tr>
+ <td>INTERVALDAY</td>
+ <td>A simple version of the interval type expressing a period of time in
days, hours, minutes, and seconds only.</td>
+ </tr>
+ <tr>
+ <td>INTERVALYEAR</td>
+ <td>A simple version of interval representing a period of time in years
and months only.</td>
+ </tr>
+ <tr>
+ <td>SMALLINT</td>
+ <td>2-byte signed integer. Supported in Drill 0.9 and later./td>
+ </tr>
+ <tr>
+ <td>TIME</td>
+ <td>Hours, minutes, seconds in the form HH:mm:ss, 24-hour based</td>
+ </tr>
+ <tr>
+ <td>TIMESTAMP</td>
+ <td>JDBC timestamp in year, month, date hour, minute, second, and optional
milliseconds format: yyyy-MM-dd HH:mm:ss.SSS</td>
+ </tr>
+ <tr>
+ <td>CHARACTER VARYING, CHARACTER, CHAR, or VARCHAR</td>
+ <td>Character string variable length. </td>
+ </tr>
+</table>
+
+<p>DATE, TIME, and TIMESTAMP store values in Coordinated Universal Time (UTC).
Currently, Drill does not support casting a TIMESTAMP with time zone, but you
can use the TO_TIMESTAMP function (link to example) in a query to use time
stamp data having a time zone.</p>
+
+<h2 id="compatibility-with-data-sources">Compatibility with Data Sources</h2>
+
+<p>The following sections describe the data type mapping between Drill and
supported data sources.</p>
+
+<ul>
+<li>HBase<br>
+None. You need to convert data as shown in <a
href="/docs/querying-hbase/">"Querying HBase."</a>.</li>
+<li>Hive<br>
+<a href="/docs/hive-to-drill-data-type-mapping">"Hive-to-Drill Data Type
Mapping"</a>.</li>
+<li>JSON<br>
+<a href="/docs/json-data-model#data-type-mapping">SQL-JSON data type
mapping</a> or to varchar in all text mode</li>
+<li>MapR-DB<br>
+<a href="/docs/mapr-db-format">The maprdb format</a> for reading (only).</li>
+<li>Parquet<br>
+<a href="/docs/parquet-format/sql-data-types-to-parquet">SQL Data Types to
Parquet</a>. </li>
+<li>Text: CSV, TSV, and other text<br>
+Implicitly casts all textual data to VARCHAR. </li>
+</ul>
+
+<p>Depending on the data format, you might need to <a
href="/docs/handling-disparate-data-types">cast or convert</a> data types
to/from these SQL types when Drill reads/writes data.</p>
+
+<h2 id="guidelines-for-using-float-and-double">Guidelines for Using Float and
Double</h2>
+
+<p>FLOAT and DOUBLE yield approximate results. These are variable-precision
numeric types. Drill does not cast/convert all values precisely to the internal
format, but instead stores approximations. Slight differences can occur in the
value stored and retrieved. The following guidelines are recommended:</p>
+
+<ul>
+<li>For conversions involving monetary calculations, for example, that require
precise results use the decimal type instead of float or double.</li>
+<li>For complex calculations or mission-critical applications, especially
those involving infinity and underflow situations, carefully consider the
limitations of type casting that involves FLOAT or DOUBLE.</li>
+<li>Equality comparisons between floating-point values can produce unexpected
results.</li>
+</ul>
+
+<p>Values of FLOAT and DOUBLE that are less than the lowest value in the range
(more negative) cause an error. Rounding can occur if the precision of an input
number is too high. </p>
+
+<h2 id="complex-data-types">Complex Data Types</h2>
+
+<p>Drill extends SQL to supoort complex and nested data structures in JSON and
Parquet files. Drill reads/writes maps and arrays from/to JSON and Parquet
files. </p>
+
+<ul>
+<li>A map is a set of name/value pairs.
+A value in a map can be a scalar type, such as string or int, or a complex
type, such as an array or another map. </li>
+<li>An array is a repeated list of values.
+A value in an array can be a scalar type, such as string or int, or an array
can be a complex type, such as a map or another array.</li>
+</ul>
+
+<p>In Drill, you do not cast a map or array to another type. </p>
+
+<p>The following example shows a JSON map having scalar values:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">phoneNumber:
+{
+ areaCode: 622,
+ number: 1567845
+}
+</code></pre></div>
+<p>The following example shows a JSON map having an array as a value:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">{
citiesLived :
+ [
+ { place : Los Angeles,
+ yearsLived : [ 1989, 1993, 1998, 2002]
+ }
+ ]
+}
+</code></pre></div>
+<p>The following example shows a JSON array having scalar values:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">yearsLived: [1990, 1993, 1998, 2008]
+</code></pre></div>
+<p>The following example shows a JSON array having complex type values:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">children:
+ [
+ { age : 10,
+ gender : Male,
+ name : Earl
+ },
+ { age : 6,
+ gender : Male,
+ name : Sam,
+ { age : 8,
+ gender : Male,
+ name : Kit
+ }
+ ]
+</code></pre></div></div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Added: drill/site/trunk/content/drill/docs/date-time-formats/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/date-time-formats/index.html?rev=1667403&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/date-time-formats/index.html (added)
+++ drill/site/trunk/content/drill/docs/date-time-formats/index.html Tue Mar 17
21:08:21 2015
@@ -0,0 +1,211 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Date/Time Formats - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Date/Time Formats</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>Using familiar date and time formats,
listed in the <a href="/docs/data-types">SQL data types table</a>, you can
construct query date and time data. You need to cast textual data to date and
time data types. The format of date, time, and timestamp text in a textual data
source needs to match the SQL query format for successful casting. </p>
+
+<h2 id="date,-time,-and-timestamp">Date, Time, and Timestamp</h2>
+
+<p>Before running a query, you can check the formatting of your dates and
times. First, create a dummy JSON file to use in the FROM clause for testing
queries as shown in the following examples.
+ {"dummy" : "data"}. </p>
+
+<p>Next, use the following literals in a SELECT statement. </p>
+
+<ul>
+<li><code>date</code></li>
+<li><code>time</code></li>
+<li><code>timestamp</code></li>
+</ul>
+
+<p>You can also use <code>interval</code> as a literal in a SELECT statement.
This usage is covered later.</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">SELECT date '2010-2-15' FROM
dfs.`/Users/drilluser/apache-drill-0.8.0/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| 2010-02-15 |
++------------+
+1 row selected (0.083 seconds)
+
+SELECT time '15:20:30' from
dfs.`/Users/drilluser/apache-drill-0.8.0/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| 15:20:30 |
++------------+
+1 row selected (0.067 seconds)
+
+SELECT timestamp '2015-03-11 6:50:08' FROM
dfs.`/Users/drilluser/apache-drill-0.8.0/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| 2015-03-11 06:50:08.0 |
++------------+
+1 row selected (0.071 seconds)
+</code></pre></div>
+<h2 id="interval-type">INTERVAL Type</h2>
+
+<p>The INTERVAL type represents a period of time. Use ISO 8601 syntax to
format a value of this type:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">P
[qty] Y [qty] M [qty] D T [qty] H [qty] M [qty] S
+
+P [qty] D T [qty] H [qty] M [qty] S
+
+P [qty] Y [qty] M
+</code></pre></div>
+<p>where:</p>
+
+<ul>
+<li>P (Period) marks the beginning of a period of time.</li>
+<li>Y follows a number of years.</li>
+<li>M follows a number of months.</li>
+<li>D follows a number of days.</li>
+<li>H follows a number of hours 0-24.</li>
+<li>M follows a number of minutes.</li>
+<li>S follows a number of seconds and optional milliseconds to the right of a
decimal point</li>
+</ul>
+
+<p>INTERVALYEAR (Year, Month) and INTERVALDAY (Day, Hours, Minutes, Seconds,
Milliseconds) are a simpler version of INTERVAL with a subset of the fields.
You do not need to specify all fields.</p>
+
+<p>The format of INTERVAL data in the data source differs from the query
format. </p>
+
+<p>You can run the dummy query described earlier to check the formatting of
the fields. The input to the following SELECT statements show how to format
INTERVAL data in the query. The output shows how to format the data in the data
source.</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">SELECT INTERVAL '1 10:20:30.123' day to second FROM
dfs.`/Users/drilluser/apache-drill-0.8.0/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| P1DT37230.123S |
++------------+
+1 row selected (0.054 seconds)
+
+SELECT INTERVAL '1-2' year to month FROM
dfs.`/Users/khahn/drill/apache-drill-0.8.0-SNAPSHOT/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| P1Y2M |
++------------+
+1 row selected (0.927 seconds)
+
+SELECT INTERVAL '1' year FROM
dfs.`/Users/khahn/drill/apache-drill-0.8.0-SNAPSHOT/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| P1Y |
++------------+
+1 row selected (0.088 seconds)
+
+SELECT INTERVAL '13' month FROM
dfs.`/Users/khahn/drill/apache-drill-0.8.0-SNAPSHOT/dummy.json`;
++------------+
+| EXPR$0 |
++------------+
+| P1Y1M |
++------------+
+1 row selected (0.076 seconds)
+</code></pre></div>
+<p>To cast INTERVAL data use the following syntax:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">CAST
(column_name AS INTERVAL)
+CAST (column_name AS INTERVAL DAY)
+CAST (column_name AS INTERVAL YEAR)
+</code></pre></div>
+<h2 id="interval-example">Interval Example</h2>
+
+<p>A JSON file contains the following objects:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">{
"INTERVALYEAR_col":"P1Y",
"INTERVALDAY_col":"P1D",
"INTERVAL_col":"P1Y1M1DT1H1M" }
+{ "INTERVALYEAR_col":"P2Y",
"INTERVALDAY_col":"P2D",
"INTERVAL_col":"P2Y2M2DT2H2M" }
+{ "INTERVALYEAR_col":"P3Y",
"INTERVALDAY_col":"P3D",
"INTERVAL_col":"P3Y3M3DT3H3M" }
+</code></pre></div>
+<p>The following CTAS statement shows how to cast text from a JSON file to
INTERVAL data types in a Parquet table:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">CREATE TABLE dfs.tmp.parquet_intervals AS
+(SELECT cast (INTERVAL_col as interval),
+ cast( INTERVALYEAR_col as interval year) INTERVALYEAR_col,
+ cast( INTERVALDAY_col as interval day) INTERVALDAY_col
+FROM `/user/root/intervals.json`);
+</code></pre></div>
+<p>Output is: </p>
+
+<p>TBD need to test in a future build.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Added:
drill/site/trunk/content/drill/docs/deploying-and-using-a-hive-udf/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/deploying-and-using-a-hive-udf/index.html?rev=1667403&view=auto
==============================================================================
---
drill/site/trunk/content/drill/docs/deploying-and-using-a-hive-udf/index.html
(added)
+++
drill/site/trunk/content/drill/docs/deploying-and-using-a-hive-udf/index.html
Tue Mar 17 21:08:21 2015
@@ -0,0 +1,131 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Deploying and Using a Hive UDF - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Deploying and Using a Hive UDF</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>If the extensive Hive functions, such as
the mathematical and date functions, which Drill supports do not meet your
needs, you can use a Hive UDF in Drill queries. Drill supports your existing
Hive scalar UDFs. You can do queries on Hive tables and access existing Hive
input/output formats, including custom serdes. Drill serves as a complement to
Hive deployments by offering low latency queries.</p>
+
+<h2 id="creating-the-udf">Creating the UDF</h2>
+
+<p>You create the JAR for a UDF to use in Drill in a conventional manner with
a few caveats, using a unique name and creating a Drill resource, covered in
this section.</p>
+
+<ol>
+<li>Use a unique name for the Hive UDF to avoid conflicts with Drill custom
functions of the same name.</li>
+<li>Create a custom Hive UDF using either of these APIs:</li>
+</ol>
+
+<ul>
+<li>Simple API: org.apache.hadoop.hive.ql.exec.UDF</li>
+<li>Complex API: org.apache.hadoop.hive.ql.udf.generic.GenericUDF
+
+<ol>
+<li>Create an empty <code>drill-module.conf</code> in the resources directory
in the Java project. </li>
+<li>Export the logic to a JAR, including the <code>drill-module.conf</code>
file in resources.</li>
+</ol></li>
+</ul>
+
+<p>The <code>drill-module.conf</code> file defines <a
href="/docs/start-up-options/">startup options</a> and makes the JAR functions
available to use in queries throughout the Hadoop cluster. After exporting the
UDF logic to a JAR file, set up the UDF in Drill. Drill users can access the
custom UDF for use in Hive queries.</p>
+
+<h2 id="setting-up-a-udf">Setting Up a UDF</h2>
+
+<p>After you export the custom UDF as a JAR, perform the UDF setup tasks so
Drill can access the UDF. The JAR needs to be available at query execution time
as a session resource, so Drill queries can refer to the UDF by its name.</p>
+
+<p>To set up the UDF:</p>
+
+<ol>
+<li>Register Hive. <a href="/docs/registering-hive/">Register a Hive storage
plugin</a> that connects Drill to a Hive data source.</li>
+<li>In Drill 0.7 and later, add the JAR for the UDF to the Drill CLASSPATH. In
earlier versions of Drill, place the JAR file in the
<code>/jars/3rdparty</code> directory of the Drill installation on all nodes
running a Drillbit.</li>
+<li>On each Drill node in the cluster, restart the Drillbit.
+<code><drill installation directory>/bin/drillbit.sh restart</code></li>
+</ol>
+
+<h2 id="using-a-udf">Using a UDF</h2>
+
+<p>Use a Hive UDF just as you would use a Drill custom function. For example,
to query using a Hive UDF named upper-to-lower that takes a column.value
argument, the SELECT statement looks something like this: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">
SELECT upper-to-lower(my_column.myvalue) FROM mytable;
+</code></pre></div></div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Modified:
drill/site/trunk/content/drill/docs/drill-default-input-format/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/drill-default-input-format/index.html?rev=1667403&r1=1667402&r2=1667403&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/drill-default-input-format/index.html
(original)
+++ drill/site/trunk/content/drill/docs/drill-default-input-format/index.html
Tue Mar 17 21:08:21 2015
@@ -127,7 +127,15 @@ steps:</p>
}
</code></pre></div></li>
</ol>
-</div>
+
+<h2 id="querying-compressed-json">Querying Compressed JSON</h2>
+
+<p>You can use Drill 0.8 and later to query compressed JSON in .gz files as
well as uncompressed files having the .json extension. First, add the gz
extension to a storage plugin, and then use that plugin to query the compressed
file.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">
"extensions": [
+ "json",
+ "gz"
+ ]
+</code></pre></div></div>
<div id="footer" class="mw">
Added: drill/site/trunk/content/drill/docs/file-system-storage-plugin/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/file-system-storage-plugin/index.html?rev=1667403&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/file-system-storage-plugin/index.html
(added)
+++ drill/site/trunk/content/drill/docs/file-system-storage-plugin/index.html
Tue Mar 17 21:08:21 2015
@@ -0,0 +1,156 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>File System Storage Plugin - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>File System Storage Plugin</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>You can register a storage plugin
instance that connects Drill to a local file
+system or a distributed file system registered in <code>core-site.xml</code>,
such as S3
+or HDFS. When you register a storage plugin instance for a file system,
+provide a unique name for the instance, and identify the type as
â<code>file</code>â. By
+default, Drill includes an instance named <code>dfs</code> that points to the
local file
+system on your machine. You can update this configuration to point to a
+distributed file system or you can create a new instance to point to a
+distributed file system.</p>
+
+<p>To register a local or a distributed file system with Apache Drill, complete
+the following steps:</p>
+
+<ol>
+<li>Navigate to <code>[http://localhost:8047](http://localhost:8047/)</code>,
and select the <strong>Storage</strong> tab.</li>
+<li>In the New Storage Plugin window, enter a unique name and then click
<strong>Create</strong>.</li>
+<li><p>In the Configuration window, provide the following configuration
information for the type of file system that you are configuring as a data
source.</p>
+
+<ol>
+<li><p>Local file system example:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">{
+ "type": "file",
+ "enabled": true,
+ "connection": "file:///",
+ "workspaces": {
+ "root": {
+ "location": "/user/max/donuts",
+ "writable": false,
+ "defaultinputformat": null
+ }
+ },
+ "formats" : {
+ "json" : {
+ "type" : "json"
+ }
+ }
+ }
+</code></pre></div></li>
+<li><p>Distributed file system example:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">{
+ "type" : "file",
+ "enabled" : true,
+ "connection" : "hdfs://10.10.30.156:8020/",
+ "workspaces" : {
+ "root : {
+ "location" : "/user/root/drill",
+ "writable" : true,
+ "defaultinputformat" : "null"
+ }
+ },
+ "formats" : {
+ "json" : {
+ "type" : "json"
+ }
+ }
+}
+</code></pre></div></li>
+</ol>
+
+<p>To connect to a Hadoop file system, you must include the IP address of the
+name node and the port number.</p></li>
+<li><p>Click <strong>Enable</strong>.</p></li>
+</ol>
+
+<p>Once you have configured a storage plugin instance for the file system, you
+can issue Drill queries against it.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Added: drill/site/trunk/content/drill/docs/getting-started-tutorial/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/getting-started-tutorial/index.html?rev=1667403&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/getting-started-tutorial/index.html
(added)
+++ drill/site/trunk/content/drill/docs/getting-started-tutorial/index.html Tue
Mar 17 21:08:21 2015
@@ -0,0 +1,167 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Getting Started Tutorial - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Getting Started Tutorial</h1>
+
+</div>
+
+<div class="int_text" align="left"><h2 id="goal">Goal</h2>
+
+<p>This tutorial covers how to query a file and a directory on your local file
+system. Files and directories are like standard SQL tables to Drill. If you
+install Drill in <a href="/docs/installing-drill-in-embedded-mode">embedded
+mode</a>, the
+installer registers and configures your file system as the <code>dfs</code>
instance.
+You can query these types of files using the default <code>dfs</code> storage
plugin:</p>
+
+<ul>
+<li>Plain text files, such as comma-separated values (CSV) or tab-separated
values (TSV) files</li>
+<li>JSON files</li>
+<li>Parquet files</li>
+</ul>
+
+<p>In this tutorial, you query plain text files using the <code>dfs</code>
storage plugin. You also create a custom storage
+plugin to simplify querying plain text files.</p>
+
+<h2 id="prerequisites">Prerequisites</h2>
+
+<p>This tutorial assumes that you installed Drill in <a
href="/docs/installing-drill-in-embedded-mode">embedded
+mode</a>. The first few lessons of the tutorial
+use a Google file of Ngram data that you download from the internet. The
+compressed Google Ngram files are 8 and 58MB. To expand the compressed files,
+you need an additional 448MB of free disk space for this exercise.</p>
+
+<p>To get started, use the SQLLine command to start the Drill command line
+interface (CLI) on Linux, Mac OS X, or Windows.</p>
+
+<h3 id="start-drill-(linux-or-mac-os-x)">Start Drill (Linux or Mac OS X)</h3>
+
+<p>To <a href="/docs/starting-stopping-drill">start Drill</a> on Linux
+or Mac OS X, use the SQLLine command.</p>
+
+<ol>
+<li>Open a terminal.</li>
+<li><p>Navigate to the Drill installation directory.</p>
+
+<p>Example: <code>$ cd ~/apache-drill-<version></code></p></li>
+<li><p>Issue the following command:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">$
bin/sqlline -u jdbc:drill:zk=local
+</code></pre></div>
+<p>The Drill prompt appears: <code>0: jdbc:drill:zk=local</code></p></li>
+</ol>
+
+<h3 id="start-drill-(windows)">Start Drill (Windows)</h3>
+
+<p>To <a href="/docs/starting-stopping-drill">start Drill</a> on
+Windows, use the SQLLine command.</p>
+
+<ol>
+<li>Open the <code>apache-drill-<version></code> folder.</li>
+<li>Open the <code>bin</code> folder, and double-click on the
<code>sqlline.bat</code> file. The Windows command prompt opens.</li>
+<li><p>At the <code>sqlline></code> prompt, issue the following command,
and then press <strong>Enter</strong>:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">!connect jdbc:drill:zk=local
+</code></pre></div>
+<p>The following prompt appears: <code>0: jdbc:drill:zk=local</code></p></li>
+</ol>
+
+<h3 id="stop-drill">Stop Drill</h3>
+
+<p>To stop Drill, issue the following command at the Drill prompt.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">
0: jdbc:drill:zk=local> !quit
+</code></pre></div>
+<p>In some cases, such as stopping while a query is in progress, this command
does not stop Drill. You need to kill the Drill process. For example, on Mac OS
X and Linux, follow
+these steps:</p>
+
+<ol>
+<li>Issue a CTRL Z to stop the query, then start Drill again. If the startup
message indicates success, skip the rest of the steps. If not, proceed to step
2.</li>
+<li><p>Search for the Drill process ID.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">$ ps
auwx | grep drill
+</code></pre></div></li>
+<li><p>Kill the process using the process number in the grep output. For
example:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">$
sudo kill -9 2674
+</code></pre></div></li>
+</ol>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
Modified:
drill/site/trunk/content/drill/docs/getting-to-know-the-drill-sandbox/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/getting-to-know-the-drill-sandbox/index.html?rev=1667403&r1=1667402&r2=1667403&view=diff
==============================================================================
---
drill/site/trunk/content/drill/docs/getting-to-know-the-drill-sandbox/index.html
(original)
+++
drill/site/trunk/content/drill/docs/getting-to-know-the-drill-sandbox/index.html
Tue Mar 17 21:08:21 2015
@@ -67,54 +67,56 @@
</div>
-<div class="int_text" align="left"><p>This section describes the configuration
of the Apache Drill system that you
-have installed and introduces the overall use case for the tutorial.</p>
+<div class="int_text" align="left"><p>This section covers key information
about the Apache Drill tutorial. After <a
href="/docs/installing-the-apache-drill-sandbox">installing the Drill
sandbox</a> and starting the sandbox, you can open another terminal window
(Linux) or Command Prompt (Windows) and use the secure shell (ssh) to connect
to the VM, assuming ssh is installed. Use the following login name and
password: mapr/mapr. For
+example:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">$ ssh
mapr@localhost -p 2222
+Password:
+Last login: Mon Sep 15 13:46:08 2014 from 10.250.0.28
+Welcome to your Mapr Demo virtual machine.
+</code></pre></div>
+<p>Using the secure shell instead of the VM interface has some advantages. You
can copy/paste commands from the tutorial and avoid mouse control problems.</p>
-<h1 id="storage-plugins-overview">Storage Plugins Overview</h1>
+<p>Drill includes SQLLine, a JDBC utility for connecting to relational
databases and executing SQL commands. After logging into the sandbox, use the
<code>sqlline</code> command to start SQLLine for executing Drill queries. </p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">[mapr@maprdemo ~]# sqlline
+sqlline version 1.1.6
+0: jdbc:drill:>
+</code></pre></div>
+<p><a href="/docs/starting-stopping-drill">Starting SQLLine outside the
sandbox</a> for use with Drill requires entering more options than are shown
here. </p>
-<p>The Hadoop cluster within the sandbox is set up with MapR-FS, MapR-DB, and
-Hive, which all serve as data sources for Drill in this tutorial. Before you
-can run queries against these data sources, Drill requires each one to be
-configured as a storage plugin. A storage plugin defines the abstraction on
-the data sources for Drill to talk to and provides interfaces to read/write
-and get metadata from the data source. Each storage plugin also exposes
-optimization rules for Drill to leverage for efficient query execution.</p>
-
-<p>Take a look at the pre-configured storage plugins by opening the Drill Web
UI.</p>
-
-<p>Feel free to skip this section and jump directly to the queries: <a
href="/docs/lession-1-learn-about-the-data-set">Lesson 1:
-Learn About the Data
-Set</a></p>
+<p>In this tutorial you query a number of data sets, including Hive and HBase,
and files on the file system, such as CSV, JSON, and Parquet files. To access
these diverse data sources, you connect Drill to storage plugins. </p>
+
+<h2 id="storage-plugin-overview">Storage Plugin Overview</h2>
+
+<p>This section describes storage plugins included in the sandbox. For general
information about Drill storage plugins, see <a
href="/docs/connect-to-data-sources">"Connect to a Data Source"</a>.
+Take a look at the pre-configured storage plugins for the sandbox by opening
the Storage tab in the Drill Web UI. Launch a web browser and go to:
<code>http://<IP address>:8047/storage</code>. For example:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">http://localhost:8046/storage
+</code></pre></div>
+<p>The control panel for managing storage plugins appear.</p>
+
+<p><img src="/docs/img/get2kno_plugin.png" alt="sandbox plugin"></p>
+
+<p>You see that the following storage plugin controls:</p>
<ul>
-<li>Launch a web browser and go to: <code>http://<IP address of the
sandbox>:8047</code></li>
-<li>Go to the Storage tab</li>
-<li>Open the configured storage plugins one at a time by clicking Update</li>
-<li>You will see the following plugins configured.</li>
+<li>cp</li>
+<li>dfs</li>
+<li>hive</li>
+<li>maprdb</li>
+<li>hbase</li>
+<li>mongo</li>
</ul>
-<h2 id="maprdb">maprdb</h2>
+<p>Click Update to look at a configuration. </p>
-<p>A storage plugin configuration for MapR-DB in the sandbox. Drill uses a
single
-storage plugin for connecting to HBase as well as MapR-DB, which is an
-enterprise grade in-Hadoop NoSQL database. In addition to the following brief
example, see the <a href="/docs/registering-hbase">Registering HBase</a> for
more
-information on how to configure Drill to query HBase.</p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">{
- "type" : "hbase",
- "enabled" : true,
- "config" : {
- "hbase.table.namespace.mappings" : "*:/tables"
- }
- }
-</code></pre></div>
-<h2 id="dfs">dfs</h2>
+<p>In some cases, the storage plugin defined for this tutorial differs from
the <a href="/docs/connect-to-data-sources">default storage plugin</a> of the
same name in a Drill installation. Typically you create a storage plugin or
customize an existing one for analyzing a particular data source. </p>
+
+<p>The tutorial uses the dfs, hive, maprdb, and hbase storage plugins. </p>
-<p>This is a storage plugin configuration for the MapR file system (MapR-FS) in
-the sandbox. The connection attribute indicates the type of distributed file
-system: in this case, MapR-FS. Drill can work with any distributed system,
-including HDFS, S3, and so on.</p>
+<h3 id="dfs">dfs</h3>
-<p>The configuration also includes a set of workspaces; each one represents a
+<p>The <code>dfs</code> storage plugin in the sandbox configures a connection
to the MapR file system (MapR-FS). </p>
+
+<p>The <code>dfs</code> storage plugin configuration in the sandbox also
includes a set of workspaces; each one represents a
location in MapR-FS:</p>
<ul>
@@ -124,20 +126,7 @@ location in MapR-FS:</p>
<li>views: a workspace for creating views</li>
</ul>
-<p>A workspace in Drill is a location where users can easily access a specific
-set of data and collaborate with each other by sharing artifacts. Users can
-create as many workspaces as they need within Drill.</p>
-
-<p>Each workspace can also be configured as âwritableâ or not, which
indicates
-whether users can write data to this location and defines the storage format
-in which the data will be written (parquet, csv, json). These attributes
-become relevant when you explore SQL commands, especially CREATE TABLE
-AS (CTAS) and CREATE VIEW.</p>
-
-<p>Drill can query files and directories directly and can detect the file
formats
-based on the file extension or the first few bits of data within the file.
-However, additional information around formats is required for Drill, such as
-delimiters for text files, which are specified in the âformatsâ section
below.</p>
+<p>The <code>dfs</code> definition includes format definitions.</p>
<div class="highlight"><pre><code class="language-text" data-lang="text">{
"type": "file",
"enabled": true,
@@ -146,56 +135,45 @@ delimiters for text files, which are spe
"root": {
"location": "/mapr/demo.mapr.com/data",
"writable": false,
- "storageformat": null
+ "defaultinputformat": null
},
"clicks": {
"location": "/mapr/demo.mapr.com/data/nested",
"writable": true,
- "storageformat": "parquet"
- },
- "logs": {
- "location": "/mapr/demo.mapr.com/data/flat",
- "writable": true,
- "storageformat": "parquet"
+ "defaultinputformat": "parquet"
},
- "views": {
- "location": "/mapr/demo.mapr.com/data/views",
- "writable": true,
- "storageformat": "parquet"
- },
+ . . .
"formats": {
- "psv": {
- "type": "text",
- "extensions": [
- "tbl"
- ],
- "delimiter": "|"
- },
- "csv": {
- "type": "text",
- "extensions": [
- "csv"
- ],
- "delimiter": ","
- },
- "tsv": {
- "type": "text",
- "extensions": [
- "tsv"
- ],
- "delimiter": "\t"
- },
- "parquet": {
- "type": "parquet"
- },
- "json": {
+ . . .
+ "csv": {
+ "type": "text",
+ "extensions": [
+ "csv"
+ ],
+ "delimiter": ","
+ },
+ . . .
+ "json": {
"type": "json"
+ }
+ }
+}
+</code></pre></div>
+<h3 id="maprdb">maprdb</h3>
+
+<p>The maprdb storage plugin is a configuration for MapR-DB in the sandbox.
You use this plugin in the sandbox to query HBase as well as MapR-DB data
because the sandbox does not include HBase services. In addition to the
following brief example, see the <a
href="/docs/hbase-storage-plugin">Registering HBase</a> for more
+information on how to configure Drill to query HBase.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">{
+ "type" : "hbase",
+ "enabled" : true,
+ "config" : {
+ "hbase.table.namespace.mappings" : "*:/tables"
+ }
}
-}}
</code></pre></div>
-<h2 id="hive">hive</h2>
+<h3 id="hive">hive</h3>
-<p>A storage plugin configuration for a Hive data warehouse within the sandbox.
+<p>The hive storage plugin is a configuration for a Hive data warehouse within
the sandbox.
Drill connects to the Hive metastore by using the configured metastore thrift
URI. Metadata for Hive tables is automatically available for users to
query.</p>
<div class="highlight"><pre><code class="language-text" data-lang="text"> {
@@ -207,88 +185,20 @@ URI. Metadata for Hive tables is automat
}
}
</code></pre></div>
-<h1 id="client-application-interfaces">Client Application Interfaces</h1>
-
-<p>Drill also provides additional application interfaces for the client tools
to
-connect and access from Drill. The interfaces include the following.</p>
-
-<h3 id="odbc/jdbc-drivers">ODBC/JDBC drivers</h3>
-
-<p>Drill provides ODBC/JDBC drivers to connect from BI tools such as Tableau,
-MicroStrategy, SQUirrel, and Jaspersoft; refer to <a
href="/docs/odbc-jdbc-interfaces/using-odbc-to-%20access-apache-drill-from-bi-tools">Using
ODBC to Access Apache
-Drill from BI Tools</a> and <a
href="/docs/odbc-jdbc-interfaces#using-jdbc-to-access-apache-drill-from-squirrel">Using
JDBC to Access Apache Drill</a> to learn
-more.</p>
-
-<h3 id="sqlline">SQLLine</h3>
-
-<p>SQLLine is a JDBC application that comes packaged with Drill. In order to
-start working with it, you can use the command line on the demo cluster to log
-in as root, then enter <code>sqlline</code>. Use <code>mapr</code> as the
login password. For
-example:</p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">$ ssh
root@localhost -p 2222
-Password:
-Last login: Mon Sep 15 13:46:08 2014 from 10.250.0.28
-Welcome to your Mapr Demo virtual machine.
-[root@maprdemo ~]# sqlline
-sqlline version 1.1.6
-0: jdbc:drill:>
-</code></pre></div>
-<h3 id="drill-web-ui">Drill Web UI</h3>
-
-<p>The Drill Web UI is a simple user interface for configuring and manage
Apache
-Drill. This UI can be launched from any of the nodes in the Drill cluster. The
-configuration for Drill includes setting up storage plugins that represent the
-data sources on which Drill performs queries. The sandbox comes with storage
-plugins configured for the Hive, HBase, MapR file system, and local file
-system.</p>
-
-<p>Users and developers can get the necessary information for tuning and
-performing diagnostics on queries, such as the list of queries executed in a
-session and detailed query plan profiles for each.</p>
-
-<p>Detailed configuration and management of Drill is out of scope for this
-tutorial.</p>
-
-<p>The Web interface for Apache Drill also provides a query UI where users can
-submit queries to Drill and observe results. Here is a screen shot of the Web
-UI for Apache Drill:</p>
-
-<p><img src="/docs/img/DrillWebUI.png" alt="drill query flow"></p>
-
-<h3 id="rest-api">REST API</h3>
-
-<p>Drill provides a simple REST API for the users to query data as well as
manage
-the system. The Web UI leverages the REST API to talk to Drill.</p>
-
-<p>This tutorial introduces sample queries that you can run by using SQLLine.
-Note that you can run the queries just as easily by launching the Drill Web
-UI. No additional installation or configuration is required.</p>
-
-<h1 id="use-case-overview">Use Case Overview</h1>
+<h2 id="use-case-overview">Use Case Overview</h2>
-<p>As you run through the queries in this tutorial, put yourself in the shoes
of
-an analyst with basic SQL skills. Let us imagine that the analyst works for an
-emerging online retail business that accepts purchases from its customers
+<p>This section describes the use case that serves as the basis for the
tutorial. Imagine being an analyst with basic SQL skills who works for an
+emerging online retail business. The business accepts purchases from its
customers
through both an established web-based interface and a new mobile
application.</p>
-<p>The analyst is data-driven and operates mostly on the business side with
-little or no interaction with the IT department. Recently the central IT team
+<p>Your job is data-driven and independent with little or no interaction with
the IT department. Recently the central IT team
has implemented a Hadoop-based infrastructure to reduce the cost of the legacy
database system, and most of the DWH/ETL workload is now handled by
-Hadoop/Hive. The master customer profile information and product catalog are
-managed in MapR-DB, which is a NoSQL database. The IT team has also started
+Hadoop/Hive. MapR DB manages the master customer profile information and
product catalog. MapR DB is a NoSQL database. The IT team has also started
acquiring clickstream data that comes from web and mobile applications. This
data is stored in Hadoop as JSON files.</p>
-<p>The analyst has a number of data sources that he could explore, but
exploring
-them in isolation is not the way to go. There are some potentially very
-interesting analytical connections between these data sources. For example, it
-would be good to be able to analyze customer records in the clickstream data
-and tie them to the master customer data in MapR DB.</p>
-
-<p>The analyst decides to explore various data sources and he chooses to do
that
-by using Apache Drill. Think about the flexibility and analytic capability of
-Apache Drill as you work through the tutorial.</p>
+<p>You have a number of data sources to explore. For example, analyzing
customer records in the clickstream data and tying them to the master customer
data in MapR DB might yield some potentially interesting analytical
connections. You decide to explore various data sources by using Apache Drill.
You need Apache Drill to provide flexibility and analytic capability.</p>
<h1 id="what's-next">What's Next</h1>
Added:
drill/site/trunk/content/drill/docs/handling-disparate-data-types/index.html
URL:
http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/handling-disparate-data-types/index.html?rev=1667403&view=auto
==============================================================================
---
drill/site/trunk/content/drill/docs/handling-disparate-data-types/index.html
(added)
+++
drill/site/trunk/content/drill/docs/handling-disparate-data-types/index.html
Tue Mar 17 21:08:21 2015
@@ -0,0 +1,408 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Handling Disparate Data Types - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class="logo"><a href="/"></a></li>
+ <li>
+ <a href="/overview/">Documentation</a>
+ <ul>
+ <li><a href="/overview/">Overview </a></li>
+ <li><a
href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes";
target="_blank">Drill in 10 Minutes</a></li>
+ <li><a href="/why/">Why Drill? </a></li>
+ <li><a href="/architecture/">Architecture</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="/community/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/community/#events">Events and Meetups</a></li>
+ <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+ <li><a href="/community/#getinvolved">Get Involved</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/DRILL/";
target="_blank">Issue Tracker</a></li>
+ <li><a href="https://github.com/apache/drill";
target="_blank">GitHub</a></li>
+ </ul>
+ </li>
+ <li><a href="/faq/">FAQ</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li style="width:30px; padding-left: 2px; padding-right:10px"><a
href="https://twitter.com/apachedrill"; target="_blank"><img
src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+ <li class="l"><span> </span></li>
+ <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Handling Disparate Data Types</h1>
+
+</div>
+
+<div class="int_text" align="left"><p><a
href="/docs/supported-date-time-data-type-formats">Previous</a><code> </code><a
href="/docs">Back to Table of
Contents</a><code> </code><a
href="/docs/lexical-structure">Next</a></p>
+
+<p>The file format of the data and queries you plan to use determine the
casting or converting you need to do, if any. When Drill reads schema-less data
into SQL tables for querying, you might need to cast one data type to another
explicitly. In some cases, Drill converts schema-less data to typed data
implicitly. In this case, you do not need to cast. Drill does not implicitly
cast HBase binary data. You use CONVERT_TO and CONVERT_FROM functions to work
with HBase data in Drill.</p>
+
+<p>The following list describes how Drill treats data types from various data
sources:</p>
+
+<ul>
+<li>HBase<br>
+No implicit casting to SQL types. Convert data to appropriate types as shown
in <a href="/docs/querying-hbase/">"Querying HBase."</a></li>
+<li>Hive<br>
+Implicitly casts Hive types to SQL types as shown in the Hive <a
href="/docs/hive-to-drill-data-type-mapping#type-mapping-example">type mapping
example</a></li>
+<li>JSON<br>
+Implicitly casts JSON data to its <a
href="/docs/json-data-model#data-type-mapping">corresponding SQL types</a> or
to VARCHAR if Drillis in all text mode. </li>
+<li>MapR-DB<br>
+Implicitly casts MapR-DB data to SQL types when you use <a
href="/docs/mapr-db-format">the maprdb format</a> for reading MapR-DB data. The
dfs storage plugin defines the format when you install Drill from the
mapr-drill package on a MapR node.</li>
+<li>Parquet<br>
+Implicitly casts Parquet data to the SQL types shown in <a
href="/docs/parquet-format/sql-data-types-to-parquet">SQL Data Types to
Parquet</a>. </li>
+<li>Text: CSV, TSV, and other text<br>
+Implicitly casts all textual data to VARCHAR.</li>
+</ul>
+
+<h2 id="implicit-casting">Implicit Casting</h2>
+
+<p>In general, Drill implicitly casts (promotes) one type to another type
based in the order of precedence, high to low, shown in the following table. A
type that has a lower precedence can be implicitly cast to type of higher
precedence. For instance, NULL can be promoted to any other type; SMALLINT can
be promoted into INT. INT cannot be promoted to SMALLINT due to possible
precision loss.</p>
+
+<p>Under certain circumstances, such as queries involving functions such as
substr and concat, Drill reverses the order of precedence and allows a cast to
VARCHAR from a type of higher precedence, such as BIGINT. Drill implicitly
casts to more data types than currently supported for explicit casting.</p>
+
+<table>
+ <tr>
+ <th>Data Types by Precedence (high to low)</th>
+ <th></th>
+ </tr>
+ <tr>
+ <td>24 INTERVAL</td>
+ <td>12 BIGINT</td>
+ </tr>
+ <tr>
+ <td>23 INTERVALYEAR</td>
+ <td>11UINT4</td>
+ </tr>
+ <tr>
+ <td>22 INTERVLADAY</td>
+ <td>10 INT</td>
+ </tr>
+ <tr>
+ <td>21 TIMESTAMPTZ</td>
+ <td>9 UINT2</td>
+ </tr>
+ <tr>
+ <td>20 TIMETZ</td>
+ <td>8 SMALLINT</td>
+ </tr>
+ <tr>
+ <td>19 TIMESTAMP</td>
+ <td>7 UINT1</td>
+ </tr>
+ <tr>
+ <td>18 DATE</td>
+ <td>6 VAR16CHAR</td>
+ </tr>
+ <tr>
+ <td>17 TIME</td>
+ <td>5 FIXED16CHAR</td>
+ </tr>
+ <tr>
+ <td>16 FLOAT8</td>
+ <td>4 VARCHAR</td>
+ </tr>
+ <tr>
+ <td>15 DECIMAL</td>
+ <td>3 FIXEDCHAR</td>
+ </tr>
+ <tr>
+ <td>14 MONEY</td>
+ <td>2 VARBINARY</td>
+ </tr>
+ <tr>
+ <td>13 UINT8</td>
+ <td>1 FIXEDBINARY</td>
+ </tr>
+ <tr>
+ <td></td>
+ <td>0 NULL</td>
+ </tr>
+</table>
+
+<h2 id="explicit-casting">Explicit Casting</h2>
+
+<p>Drill supports a number of functions to cast and convert compatible data
types:</p>
+
+<ul>
+<li>CAST<br>
+Casts textual data from one data type to another.</li>
+<li>CONVERT_TO and CONVERT_FROM<br>
+Converts data, including binary data, from one data type to another.</li>
+<li>TO_CHAR
+Converts a TIMESTAMP, INTERVAL, INTEGER, DOUBLE, or DECIMAL to a string.</li>
+<li>TO_DATE
+Converts a string to DATE.</li>
+<li>TO_NUMBER
+Converts a string to a DECIMAL.</li>
+<li>TO_TIMESTAMP
+Converts a string to TIMESTAMP.</li>
+</ul>
+
+<h3 id="using-cast">Using CAST</h3>
+
+<p>Embed a CAST function in a query using this syntax:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">cast
<expression> AS <data type>
+</code></pre></div>
+<ul>
+<li>expression<br>
+An entity that has single data value, such as a column name, of the data type
you want to cast to a different type</li>
+<li>data type<br>
+The target data type, such as INTEGER or DATE</li>
+</ul>
+
+<p>Example: Inspect INTEGER data and cast the data to the DECIMAL type</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">SELECT c_row, c_int FROM mydata WHERE c_row = 9;
+
+c_row | c_int
+------+------------
+ 9 | -2147483648
+(1 row)
+
+SELECT c_row, CAST(c_int AS DECIMAL(28,8)) FROM my_data WHERE c_row = 9;
+
+c_row | c_int
+------+---------------------
+9 | -2147483648.00000000
+(1 row)
+</code></pre></div>
+<p>If the SELECT statement includes a WHERE clause that compares a column of
an unknown data type, cast both the value of the column and the comparison
value in the WHERE clause. For example:</p>
+<div class="highlight"><pre><code class="language-text"
data-lang="text">SELECT c_row, CAST(c_int AS DECIMAL(28,8)) FROM mydata WHERE
CAST(c_int AS CECIMAL(28,8)) > -3.0
+</code></pre></div>
+<p>Although you can use CAST to handle binary data, CONVERT_TO and
CONVERT_FROM are recommended for these conversions.</p>
+
+<p>The following table shows data types that you can cast to from other data
types.</p>
+
+<table>
+ <tr>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ <th></th>
+ </tr>
+ <tr>
+ <td>From:</td>
+ <td>SMALLINT</td>
+ <td>INT</td>
+ <td>BIGINT/UINT</td>
+ <td>DECIMAL</td>
+ <td>FLOAT4</td>
+ <td>FLOAT8</td>
+ <td>FIXEDCHAR</td>
+ <td>FIXEDBINARY</td>
+ <td>VARCHAR</td>
+ <td>VARBINARY</td>
+ </tr>
+
+ <tr>
+ <td>SMALLINT</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ </tr>
+ <tr>
+ <td>INT</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ </tr>
+ <tr>
+ <td>BIGINT/UINT</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ </tr>
+ <tr>
+
+ <tr>
+ <td>DECIMAL</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ </tr>
+
+ <tr>
+ <td>FLOAT8</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>no</td>
+ </tr>
+ <tr>
+ <td>FLOAT4</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>no</td>
+ </tr>
+
+ <tr>
+ <td>FIXEDCHAR</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ </tr>
+ <tr>
+ <td>FIXEDBINARY</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>yes</td>
+ </tr>
+ <tr>
+ <td>VARCHAR</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ </tr>
+ <tr>
+ <td>VARBINARY</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ <td>yes</td>
+ <td>yes</td>
+ <td>no</td>
+ </tr>
+
+</table>
+
+<h3 id="using-convert_to-and-convert_from">Using CONVERT_TO and
CONVERT_FROM</h3>
+
+<p>To query HBase data in Drill, convert every column of an HBase table
to/from byte arrays from/to an <a href="/docs/data-types/">SQL data type</a>
that Drill supports when writing/reading data. For examples of how to use these
functions, see <a
href="/docs/sql-functions#convert-and-cast-functions">"Convert and Cast
Functions".</a></p>
+
+<h2 id="handling-textual-data">Handling Textual Data</h2>
+
+<p>In a textual file, such as CSV, Drill interprets every field as a VARCHAR,
as previously mentioned. In addition to using the CAST function, you can also
use <a href="link">to_char</a>, <a href="line">to_date</a>, <a
href="link">to_number</a>, and <a href="link">to_timestamp</a>. If the SELECT
statement includes a WHERE clause that compares a column of an unknown data
type, cast both the value of the column and the comparison value in the WHERE
clause.</p>
+
+<h2 id="all-text-mode-option">All text mode option</h2>
+
+<p>All text mode is a system option for controlling how Drill implicitly casts
JSON data. When reading numerical values from a JSON file, Drill implicitly
casts a number to the DOUBLE or BIGINT type depending on the presence or
absence a decimal point. If some numbers in a JSON map or array appear with and
without a decimal point, such as 0 and 0.0, Drill throws a schema change error.
To prevent Drill from attempting to read such data, <a
href="/docs/json-data-model#handling-type-differences">set all_text_mode</a> to
true. In all text mode, Drill implicitly casts JSON data to VARCHAR, which you
can subsequently cast to desired types.</p>
+
+<p>Drill reads numbers without decimal point as BIGINT values by default. The
range of BIGINT is -9223372036854775808 to 9223372036854775807. A BIGINT result
outside this range produces an error. Use <code>all_text_mode</code> to select
data as VARCHAR and then cast the data to a numerical type.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>
