Repository: drill-site Updated Branches: refs/heads/asf-site 7af688205 -> debb5d3df
website update Project: http://git-wip-us.apache.org/repos/asf/drill-site/repo Commit: http://git-wip-us.apache.org/repos/asf/drill-site/commit/debb5d3d Tree: http://git-wip-us.apache.org/repos/asf/drill-site/tree/debb5d3d Diff: http://git-wip-us.apache.org/repos/asf/drill-site/diff/debb5d3d Branch: refs/heads/asf-site Commit: debb5d3dfe71372fa48fbd94faa972b2cbd1f83c Parents: 7af6882 Author: Kris Hahn <[email protected]> Authored: Tue Dec 15 18:10:35 2015 -0800 Committer: Kris Hahn <[email protected]> Committed: Tue Dec 15 18:10:35 2015 -0800 ---------------------------------------------------------------------- .../index.html | 34 ++++++++++++++++---- docs/storage-plugin-registration/index.html | 4 +-- docs/text-files-csv-tsv-psv/index.html | 17 +++++++++- feed.xml | 4 +-- 4 files changed, 47 insertions(+), 12 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/drill-site/blob/debb5d3d/docs/configuration-options-introduction/index.html ---------------------------------------------------------------------- diff --git a/docs/configuration-options-introduction/index.html b/docs/configuration-options-introduction/index.html index c950109..daa0590 100644 --- a/docs/configuration-options-introduction/index.html +++ b/docs/configuration-options-introduction/index.html @@ -1071,7 +1071,7 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <tr> <td>drill.exec.functions.cast_empty_string_to_null</td> <td>FALSE</td> -<td>Not supported in this release.</td> +<td>In a text file, treat empty fields as NULL values instead of empty string.</td> </tr> <tr> <td>drill.exec.storage.file.partition.column.label</td> @@ -1079,6 +1079,11 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <td>The column label for directory levels in results of queries of files in a directory. Accepts a string input.</td> </tr> <tr> +<td>exec.enable_union_type</td> +<td>false</td> +<td>Enable support for Avro union type.</td> +</tr> +<tr> <td>exec.errors.verbose</td> <td>FALSE</td> <td>Toggles verbose output of executable error messages</td> @@ -1284,11 +1289,6 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <td>A heuristic value for influencing the size of the hash aggregation table.</td> </tr> <tr> -<td>planner.memory_limit</td> -<td>268435456 bytes</td> -<td>Defines the maximum amount of direct memory allocated to a query for planning. When multiple queries run concurrently, each query is allocated the amount of memory set by this parameter.Increase the value of this parameter and rerun the query if partition pruning failed due to insufficient memory.</td> -</tr> -<tr> <td>planner.memory.max_query_memory_per_node</td> <td>2147483648 bytes</td> <td>Sets the maximum estimate of memory for a query per node in bytes. If the estimate is too low, Drill re-plans the query without memory-constrained operators.</td> @@ -1299,6 +1299,11 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <td>Extra query memory per node for non-blocking operators. This option is currently used only for memory estimation. Range: 0-2048 MB</td> </tr> <tr> +<td>planner.memory_limit</td> +<td>268435456 bytes</td> +<td>Defines the maximum amount of direct memory allocated to a query for planning. When multiple queries run concurrently, each query is allocated the amount of memory set by this parameter.Increase the value of this parameter and rerun the query if partition pruning failed due to insufficient memory.</td> +</tr> +<tr> <td>planner.nestedloopjoin_factor</td> <td>100</td> <td>A heuristic value for influencing the nested loop join.</td> @@ -1339,6 +1344,16 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <td>Same as max per node but applies to the query as executed by the entire cluster. For example, this value might be the number of active Drillbits, or a higher number to return results faster.</td> </tr> <tr> +<td>security.admin.user_groups</td> +<td>n/a</td> +<td>Unsupported as of 1.4. A comma-separated list of administrator groups for Web Console security.</td> +</tr> +<tr> +<td>security.admin.users</td> +<td><a name></td> +<td>Unsupported as of 1.4. A comma-separated list of user names who you want to give administrator privileges.</td> +</tr> +<tr> <td>store.format</td> <td>parquet</td> <td>Output format for data written to tables with the CREATE TABLE AS (CTAS) command. Allowed values are parquet, json, psv, csv, or tsv.</td> @@ -1356,7 +1371,7 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <tr> <td>store.json.extended_types</td> <td>FALSE</td> -<td>Turns on special JSON structures that Drill serializes for storing more type information than the <a href="http://docs.mongodb.org/manual/reference/mongodb-extended-json/";>four basic JSON types</a>.</td> +<td>Turns on special JSON structures that Drill serializes for storing more type information than the four basic JSON types.</td> </tr> <tr> <td>store.json.read_numbers_as_double</td> @@ -1389,6 +1404,11 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d <td>For internal use. Do not change.</td> </tr> <tr> +<td>store.parquet.dictionary.page-size</td> +<td>1048576</td> +<td></td> +</tr> +<tr> <td>store.parquet.use_new_reader</td> <td>FALSE</td> <td>Not supported in this release.</td> http://git-wip-us.apache.org/repos/asf/drill-site/blob/debb5d3d/docs/storage-plugin-registration/index.html ---------------------------------------------------------------------- diff --git a/docs/storage-plugin-registration/index.html b/docs/storage-plugin-registration/index.html index 0337ea8..0573f3f 100644 --- a/docs/storage-plugin-registration/index.html +++ b/docs/storage-plugin-registration/index.html @@ -1076,9 +1076,9 @@ Provides a connection to MongoDB data.</li> <h2 id="storage-plugin-configuration-persistance">Storage Plugin Configuration Persistance</h2> -<p>Drill saves storage plugin configurations in a temporary directory (embedded mode) or in ZooKeeper (distributed mode). For example, on Mac OS X, Drill uses <code>/tmp/drill/sys.storage_plugins</code> to store storage plugin configurations. The temporary directory clears when you quit the Drill shell. To save your storage plugin configurations from one session to the next, set the following option in the <code>drill-override.conf</code> file if you are running Drill in embedded mode.</p> +<p>Drill saves storage plugin configurations in a temporary directory (embedded mode) or in ZooKeeper (distributed mode). For example, on Mac OS X, Drill uses <code>/tmp/drill/sys.storage_plugins</code> to store storage plugin configurations. The temporary directory clears when you reboot. Copy storage plugin configurations to a secure location to save them when you run drill in embedded mode.</p> -<p><code>drill.exec.sys.store.provider.local.path = "/mypath"</code></p> +<!-- `drill.exec.sys.store.provider.local.path = "/mypath"` --> <!-- Enabling authorization to protect this data through the Web Console and REST API does not include protection for the data in the tmp directory or in ZooKeeper. http://git-wip-us.apache.org/repos/asf/drill-site/blob/debb5d3d/docs/text-files-csv-tsv-psv/index.html ---------------------------------------------------------------------- diff --git a/docs/text-files-csv-tsv-psv/index.html b/docs/text-files-csv-tsv-psv/index.html index 3eb5ec3..0532d3b 100644 --- a/docs/text-files-csv-tsv-psv/index.html +++ b/docs/text-files-csv-tsv-psv/index.html @@ -1070,7 +1070,22 @@ FROM `users.csv1`; </code></pre></div> <h3 id="cast-data">Cast data</h3> -<p>You can also improve performance by casting the VARCHAR data to INT, FLOAT, DATETIME, and so on when you read the data from a text file. Drill performs better reading fixed-width than reading VARCHAR data. </p> +<p>You can also improve performance by casting the VARCHAR data in a text file to INT, FLOAT, DATETIME, and so on when you read the data from a text file. Drill performs better reading fixed-width than reading VARCHAR data. </p> + +<p>Text files that include empty strings might produce unacceptable results. Common ways to deal with empty strings are:</p> + +<ul> +<li>Set the drill.exec.functions.cast_empty_string_to_null SESSION/SYSTEM option to true.<br></li> +<li><p>Use a case statement to cast empty strings to values you want. For example, create a Parquet table named test from a CSV file named test.csv, and cast empty strings in the CSV to null in any column the empty string appears: </p> +<div class="highlight"><pre><code class="language-text" data-lang="text"> CREATE TABLE test AS SELECT + case when COLUMNS[0] = '' then CAST(NULL AS INTEGER) else CAST(COLUMNS[0] AS INTEGER) end AS c1, + case when COLUMNS[1] = '' then CAST(NULL AS VARCHAR(20)) else CAST(COLUMNS[1] AS VARCHAR(20)) end AS c2, + case when COLUMNS[2] = '' then CAST(NULL AS DOUBLE) else CAST(COLUMNS[2] AS DOUBLE) end AS c3, + case when COLUMNS[3] = '' then CAST(NULL AS DATE) else CAST(COLUMNS[3] AS DATE) end AS c4, + case when COLUMNS[4] = '' then CAST(NULL AS VARCHAR(20)) else CAST(COLUMNS[4] AS VARCHAR(20)) end AS c5 + FROM `test.csv`; +</code></pre></div></li> +</ul> <h3 id="use-a-distributed-file-system">Use a Distributed File System</h3> http://git-wip-us.apache.org/repos/asf/drill-site/blob/debb5d3d/feed.xml ---------------------------------------------------------------------- diff --git a/feed.xml b/feed.xml index 9c9296d..14f59f4 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ </description> <link>/</link> <atom:link href="/feed.xml" rel="self" type="application/rss+xml"/> - <pubDate>Tue, 15 Dec 2015 11:15:37 -0800</pubDate> - <lastBuildDate>Tue, 15 Dec 2015 11:15:37 -0800</lastBuildDate> + <pubDate>Tue, 15 Dec 2015 18:07:55 -0800</pubDate> + <lastBuildDate>Tue, 15 Dec 2015 18:07:55 -0800</lastBuildDate> <generator>Jekyll v2.4.0</generator> <item>
