http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/configuration_reference.html ---------------------------------------------------------------------- diff --git a/docs/configuration_reference.html b/docs/configuration_reference.html index 878d377..586d5f2 100644 --- a/docs/configuration_reference.html +++ b/docs/configuration_reference.html @@ -1147,7 +1147,7 @@ configuration tasks.</p> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">runtime,advanced,stable</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced,runtime,stable</p></td> </tr> </tbody> </table> @@ -2240,7 +2240,7 @@ configuration tasks.</p> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">runtime,advanced,stable</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced,runtime,stable</p></td> </tr> </tbody> </table> @@ -2988,6 +2988,84 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-master_missed_heartbeats_before_rejecting_snapshot_scans"><a class="link" href="#kudu-master_missed_heartbeats_before_rejecting_snapshot_scans"><code>--missed_heartbeats_before_rejecting_snapshot_scans</code></a></h4> +<div class="paragraph"> +<p>The maximum raft heartbeat periods since the tablet has seen safe time advanced before refusing scans at snapshots that aren’t yet safe and forcing clients to try again.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">double</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1.5</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-master_safe_time_advancement_without_writes"><a class="link" href="#kudu-master_safe_time_advancement_without_writes"><code>--safe_time_advancement_without_writes</code></a></h4> +<div class="paragraph"> +<p>Whether to enable the advancement of "safe" time in the absense of write operations</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-master_safe_time_max_lag_ms"><a class="link" href="#kudu-master_safe_time_max_lag_ms"><code>--safe_time_max_lag_ms</code></a></h4> +<div class="paragraph"> +<p>The maximum amount of time we allow safe time to lag behind the requested timestampbefore forcing the client to retry, in milliseconds.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>30000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-master_block_coalesce_close"><a class="link" href="#kudu-master_block_coalesce_close"><code>--block_coalesce_close</code></a></h4> <div class="paragraph"> <p>Coalesce synchronization of data during CloseBlocks()</p> @@ -3014,6 +3092,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-master_block_manager_max_open_files"><a class="link" href="#kudu-master_block_manager_max_open_files"><code>--block_manager_max_open_files</code></a></h4> +<div class="paragraph"> +<p>Maximum number of open file descriptors to be used for data blocks. If 0, there is no limit. If -1, Kudu will use half of its resource limit as per getrlimit(). This is a soft limit.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-master_fs_data_dirs_full_disk_cache_seconds"><a class="link" href="#kudu-master_fs_data_dirs_full_disk_cache_seconds"><code>--fs_data_dirs_full_disk_cache_seconds</code></a></h4> <div class="paragraph"> <p>Number of seconds we cache the full-disk status in the block manager. During this time, writes to the corresponding root path will not be attempted.</p> @@ -3092,6 +3196,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-master_log_container_max_blocks"><a class="link" href="#kudu-master_log_container_max_blocks"><code>--log_container_max_blocks</code></a></h4> +<div class="paragraph"> +<p>Maximum number of blocks (soft) of a log container. Use 0 for no limit. Use -1 for no limit except in the case of a kernel bug with hole punching on ext4 (see KUDU-1508 for details).</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-master_log_container_max_size"><a class="link" href="#kudu-master_log_container_max_size"><code>--log_container_max_size</code></a></h4> <div class="paragraph"> <p>Maximum size (soft) of a log container</p> @@ -3290,7 +3420,7 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>3600000</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>300000</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> @@ -4730,6 +4860,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-master_scanner_max_wait_ms"><a class="link" href="#kudu-master_scanner_max_wait_ms"><code>--scanner_max_wait_ms</code></a></h4> +<div class="paragraph"> +<p>The maximum amount of time (in milliseconds) we’ll hang a scanner thread waiting for safe time to advance or transactions to commit, even if its deadline allows waiting longer.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-master_num_tablets_to_copy_simultaneously"><a class="link" href="#kudu-master_num_tablets_to_copy_simultaneously"><code>--num_tablets_to_copy_simultaneously</code></a></h4> +<div class="paragraph"> +<p>Number of threads available to copy tablets from remote servers.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-master_num_tablets_to_open_simultaneously"><a class="link" href="#kudu-master_num_tablets_to_open_simultaneously"><code>--num_tablets_to_open_simultaneously</code></a></h4> <div class="paragraph"> <p>Number of threads available to open tablets during startup. If this is set to 0 (the default), then the number of bootstrap threads will be set based on the number of data directories. If the data directories are on some very fast storage device such as SSD or a RAID array, it may make sense to manually tune this.</p> @@ -4860,6 +5042,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-master_file_cache_expiry_period_ms"><a class="link" href="#kudu-master_file_cache_expiry_period_ms"><code>--file_cache_expiry_period_ms</code></a></h4> +<div class="paragraph"> +<p>Period of time (in ms) between removing expired file cache descriptors</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>60000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-master_disable_core_dumps"><a class="link" href="#kudu-master_disable_core_dumps"><code>--disable_core_dumps</code></a></h4> <div class="paragraph"> <p>Disable core dumps when this process crashes.</p> @@ -4886,6 +5094,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-master_log_redact_user_data"><a class="link" href="#kudu-master_log_redact_user_data"><code>--log_redact_user_data</code></a></h4> +<div class="paragraph"> +<p>Whether log and error messages will have row data redacted.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-master_max_log_files"><a class="link" href="#kudu-master_max_log_files"><code>--max_log_files</code></a></h4> +<div class="paragraph"> +<p>Maximum number of log files to retain per severity level. The most recent log files are retained. If set to 0, all log files are retained.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-master_log_target_replay_size_mb"><a class="link" href="#kudu-master_log_target_replay_size_mb"><code>--log_target_replay_size_mb</code></a></h4> <div class="paragraph"> <p>The target maximum size of logs to be replayed at startup. If a tablet has in-memory operations that are causing more than this size of logs to be retained, then the maintenance manager will prioritize flushing these operations to disk.</p> @@ -5856,6 +6116,84 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-tserver_missed_heartbeats_before_rejecting_snapshot_scans"><a class="link" href="#kudu-tserver_missed_heartbeats_before_rejecting_snapshot_scans"><code>--missed_heartbeats_before_rejecting_snapshot_scans</code></a></h4> +<div class="paragraph"> +<p>The maximum raft heartbeat periods since the tablet has seen safe time advanced before refusing scans at snapshots that aren’t yet safe and forcing clients to try again.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">double</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1.5</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-tserver_safe_time_advancement_without_writes"><a class="link" href="#kudu-tserver_safe_time_advancement_without_writes"><code>--safe_time_advancement_without_writes</code></a></h4> +<div class="paragraph"> +<p>Whether to enable the advancement of "safe" time in the absense of write operations</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-tserver_safe_time_max_lag_ms"><a class="link" href="#kudu-tserver_safe_time_max_lag_ms"><code>--safe_time_max_lag_ms</code></a></h4> +<div class="paragraph"> +<p>The maximum amount of time we allow safe time to lag behind the requested timestampbefore forcing the client to retry, in milliseconds.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>30000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-tserver_block_coalesce_close"><a class="link" href="#kudu-tserver_block_coalesce_close"><code>--block_coalesce_close</code></a></h4> <div class="paragraph"> <p>Coalesce synchronization of data during CloseBlocks()</p> @@ -5882,6 +6220,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-tserver_block_manager_max_open_files"><a class="link" href="#kudu-tserver_block_manager_max_open_files"><code>--block_manager_max_open_files</code></a></h4> +<div class="paragraph"> +<p>Maximum number of open file descriptors to be used for data blocks. If 0, there is no limit. If -1, Kudu will use half of its resource limit as per getrlimit(). This is a soft limit.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-tserver_fs_data_dirs_full_disk_cache_seconds"><a class="link" href="#kudu-tserver_fs_data_dirs_full_disk_cache_seconds"><code>--fs_data_dirs_full_disk_cache_seconds</code></a></h4> <div class="paragraph"> <p>Number of seconds we cache the full-disk status in the block manager. During this time, writes to the corresponding root path will not be attempted.</p> @@ -5960,6 +6324,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-tserver_log_container_max_blocks"><a class="link" href="#kudu-tserver_log_container_max_blocks"><code>--log_container_max_blocks</code></a></h4> +<div class="paragraph"> +<p>Maximum number of blocks (soft) of a log container. Use 0 for no limit. Use -1 for no limit except in the case of a kernel bug with hole punching on ext4 (see KUDU-1508 for details).</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-tserver_log_container_max_size"><a class="link" href="#kudu-tserver_log_container_max_size"><code>--log_container_max_size</code></a></h4> <div class="paragraph"> <p>Maximum size (soft) of a log container</p> @@ -7338,6 +7728,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-tserver_scanner_max_wait_ms"><a class="link" href="#kudu-tserver_scanner_max_wait_ms"><code>--scanner_max_wait_ms</code></a></h4> +<div class="paragraph"> +<p>The maximum amount of time (in milliseconds) we’ll hang a scanner thread waiting for safe time to advance or transactions to commit, even if its deadline allows waiting longer.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-tserver_num_tablets_to_copy_simultaneously"><a class="link" href="#kudu-tserver_num_tablets_to_copy_simultaneously"><code>--num_tablets_to_copy_simultaneously</code></a></h4> +<div class="paragraph"> +<p>Number of threads available to copy tablets from remote servers.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-tserver_num_tablets_to_open_simultaneously"><a class="link" href="#kudu-tserver_num_tablets_to_open_simultaneously"><code>--num_tablets_to_open_simultaneously</code></a></h4> <div class="paragraph"> <p>Number of threads available to open tablets during startup. If this is set to 0 (the default), then the number of bootstrap threads will be set based on the number of data directories. If the data directories are on some very fast storage device such as SSD or a RAID array, it may make sense to manually tune this.</p> @@ -7468,6 +7910,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-tserver_file_cache_expiry_period_ms"><a class="link" href="#kudu-tserver_file_cache_expiry_period_ms"><code>--file_cache_expiry_period_ms</code></a></h4> +<div class="paragraph"> +<p>Period of time (in ms) between removing expired file cache descriptors</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>60000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-tserver_disable_core_dumps"><a class="link" href="#kudu-tserver_disable_core_dumps"><code>--disable_core_dumps</code></a></h4> <div class="paragraph"> <p>Disable core dumps when this process crashes.</p> @@ -7494,6 +7962,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect3"> +<h4 id="kudu-tserver_log_redact_user_data"><a class="link" href="#kudu-tserver_log_redact_user_data"><code>--log_redact_user_data</code></a></h4> +<div class="paragraph"> +<p>Whether log and error messages will have row data redacted.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> +<h4 id="kudu-tserver_max_log_files"><a class="link" href="#kudu-tserver_max_log_files"><code>--max_log_files</code></a></h4> +<div class="paragraph"> +<p>Maximum number of log files to retain per severity level. The most recent log files are retained. If set to 0, all log files are retained.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect3"> <h4 id="kudu-tserver_log_target_replay_size_mb"><a class="link" href="#kudu-tserver_log_target_replay_size_mb"><code>--log_target_replay_size_mb</code></a></h4> <div class="paragraph"> <p>The target maximum size of logs to be replayed at startup. If a tablet has in-memory operations that are causing more than this size of logs to be retained, then the maintenance manager will prioritize flushing these operations to disk.</p> @@ -7892,6 +8412,10 @@ for informational purposes only. They are subject to change or be removed withou </li> <li> + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> + <a href="export_control.html">Export Control Notice</a> </li> </ul> @@ -7901,7 +8425,7 @@ for informational purposes only. They are subject to change or be removed withou </div> <footer class="footer"> <p class="small"> - Copyright © 2016 The Apache Software Foundation. Last updated 2016-11-21 07:26:08 PST + Copyright © 2016 The Apache Software Foundation. Last updated 2017-01-18 10:16:01 PST </p> </footer> </div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/configuration_reference_unsupported.html ---------------------------------------------------------------------- diff --git a/docs/configuration_reference_unsupported.html b/docs/configuration_reference_unsupported.html index 558a6a0..1b6bfeb 100644 --- a/docs/configuration_reference_unsupported.html +++ b/docs/configuration_reference_unsupported.html @@ -795,6 +795,84 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-master_missed_heartbeats_before_rejecting_snapshot_scans"><a class="link" href="#kudu-master_missed_heartbeats_before_rejecting_snapshot_scans"><code>--missed_heartbeats_before_rejecting_snapshot_scans</code></a></h3> +<div class="paragraph"> +<p>The maximum raft heartbeat periods since the tablet has seen safe time advanced before refusing scans at snapshots that aren’t yet safe and forcing clients to try again.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">double</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1.5</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-master_safe_time_advancement_without_writes"><a class="link" href="#kudu-master_safe_time_advancement_without_writes"><code>--safe_time_advancement_without_writes</code></a></h3> +<div class="paragraph"> +<p>Whether to enable the advancement of "safe" time in the absense of write operations</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-master_safe_time_max_lag_ms"><a class="link" href="#kudu-master_safe_time_max_lag_ms"><code>--safe_time_max_lag_ms</code></a></h3> +<div class="paragraph"> +<p>The maximum amount of time we allow safe time to lag behind the requested timestampbefore forcing the client to retry, in milliseconds.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>30000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-master_block_coalesce_close"><a class="link" href="#kudu-master_block_coalesce_close"><code>--block_coalesce_close</code></a></h3> <div class="paragraph"> <p>Coalesce synchronization of data during CloseBlocks()</p> @@ -821,6 +899,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-master_block_manager_max_open_files"><a class="link" href="#kudu-master_block_manager_max_open_files"><code>--block_manager_max_open_files</code></a></h3> +<div class="paragraph"> +<p>Maximum number of open file descriptors to be used for data blocks. If 0, there is no limit. If -1, Kudu will use half of its resource limit as per getrlimit(). This is a soft limit.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-master_fs_data_dirs_full_disk_cache_seconds"><a class="link" href="#kudu-master_fs_data_dirs_full_disk_cache_seconds"><code>--fs_data_dirs_full_disk_cache_seconds</code></a></h3> <div class="paragraph"> <p>Number of seconds we cache the full-disk status in the block manager. During this time, writes to the corresponding root path will not be attempted.</p> @@ -899,6 +1003,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-master_log_container_max_blocks"><a class="link" href="#kudu-master_log_container_max_blocks"><code>--log_container_max_blocks</code></a></h3> +<div class="paragraph"> +<p>Maximum number of blocks (soft) of a log container. Use 0 for no limit. Use -1 for no limit except in the case of a kernel bug with hole punching on ext4 (see KUDU-1508 for details).</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-master_log_container_max_size"><a class="link" href="#kudu-master_log_container_max_size"><code>--log_container_max_size</code></a></h3> <div class="paragraph"> <p>Maximum size (soft) of a log container</p> @@ -1097,7 +1227,7 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>3600000</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>300000</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> @@ -2537,6 +2667,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-master_scanner_max_wait_ms"><a class="link" href="#kudu-master_scanner_max_wait_ms"><code>--scanner_max_wait_ms</code></a></h3> +<div class="paragraph"> +<p>The maximum amount of time (in milliseconds) we’ll hang a scanner thread waiting for safe time to advance or transactions to commit, even if its deadline allows waiting longer.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-master_num_tablets_to_copy_simultaneously"><a class="link" href="#kudu-master_num_tablets_to_copy_simultaneously"><code>--num_tablets_to_copy_simultaneously</code></a></h3> +<div class="paragraph"> +<p>Number of threads available to copy tablets from remote servers.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-master_num_tablets_to_open_simultaneously"><a class="link" href="#kudu-master_num_tablets_to_open_simultaneously"><code>--num_tablets_to_open_simultaneously</code></a></h3> <div class="paragraph"> <p>Number of threads available to open tablets during startup. If this is set to 0 (the default), then the number of bootstrap threads will be set based on the number of data directories. If the data directories are on some very fast storage device such as SSD or a RAID array, it may make sense to manually tune this.</p> @@ -2667,6 +2849,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-master_file_cache_expiry_period_ms"><a class="link" href="#kudu-master_file_cache_expiry_period_ms"><code>--file_cache_expiry_period_ms</code></a></h3> +<div class="paragraph"> +<p>Period of time (in ms) between removing expired file cache descriptors</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>60000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-master_disable_core_dumps"><a class="link" href="#kudu-master_disable_core_dumps"><code>--disable_core_dumps</code></a></h3> <div class="paragraph"> <p>Disable core dumps when this process crashes.</p> @@ -2693,6 +2901,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-master_log_redact_user_data"><a class="link" href="#kudu-master_log_redact_user_data"><code>--log_redact_user_data</code></a></h3> +<div class="paragraph"> +<p>Whether log and error messages will have row data redacted.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-master_max_log_files"><a class="link" href="#kudu-master_max_log_files"><code>--max_log_files</code></a></h3> +<div class="paragraph"> +<p>Maximum number of log files to retain per severity level. The most recent log files are retained. If set to 0, all log files are retained.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-master_log_target_replay_size_mb"><a class="link" href="#kudu-master_log_target_replay_size_mb"><code>--log_target_replay_size_mb</code></a></h3> <div class="paragraph"> <p>The target maximum size of logs to be replayed at startup. If a tablet has in-memory operations that are causing more than this size of logs to be retained, then the maintenance manager will prioritize flushing these operations to disk.</p> @@ -3665,6 +3925,84 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-tserver_missed_heartbeats_before_rejecting_snapshot_scans"><a class="link" href="#kudu-tserver_missed_heartbeats_before_rejecting_snapshot_scans"><code>--missed_heartbeats_before_rejecting_snapshot_scans</code></a></h3> +<div class="paragraph"> +<p>The maximum raft heartbeat periods since the tablet has seen safe time advanced before refusing scans at snapshots that aren’t yet safe and forcing clients to try again.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">double</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1.5</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-tserver_safe_time_advancement_without_writes"><a class="link" href="#kudu-tserver_safe_time_advancement_without_writes"><code>--safe_time_advancement_without_writes</code></a></h3> +<div class="paragraph"> +<p>Whether to enable the advancement of "safe" time in the absense of write operations</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-tserver_safe_time_max_lag_ms"><a class="link" href="#kudu-tserver_safe_time_max_lag_ms"><code>--safe_time_max_lag_ms</code></a></h3> +<div class="paragraph"> +<p>The maximum amount of time we allow safe time to lag behind the requested timestampbefore forcing the client to retry, in milliseconds.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>30000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-tserver_block_coalesce_close"><a class="link" href="#kudu-tserver_block_coalesce_close"><code>--block_coalesce_close</code></a></h3> <div class="paragraph"> <p>Coalesce synchronization of data during CloseBlocks()</p> @@ -3691,6 +4029,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-tserver_block_manager_max_open_files"><a class="link" href="#kudu-tserver_block_manager_max_open_files"><code>--block_manager_max_open_files</code></a></h3> +<div class="paragraph"> +<p>Maximum number of open file descriptors to be used for data blocks. If 0, there is no limit. If -1, Kudu will use half of its resource limit as per getrlimit(). This is a soft limit.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-tserver_fs_data_dirs_full_disk_cache_seconds"><a class="link" href="#kudu-tserver_fs_data_dirs_full_disk_cache_seconds"><code>--fs_data_dirs_full_disk_cache_seconds</code></a></h3> <div class="paragraph"> <p>Number of seconds we cache the full-disk status in the block manager. During this time, writes to the corresponding root path will not be attempted.</p> @@ -3769,6 +4133,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-tserver_log_container_max_blocks"><a class="link" href="#kudu-tserver_log_container_max_blocks"><code>--log_container_max_blocks</code></a></h3> +<div class="paragraph"> +<p>Maximum number of blocks (soft) of a log container. Use 0 for no limit. Use -1 for no limit except in the case of a kernel bug with hole punching on ext4 (see KUDU-1508 for details).</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-tserver_log_container_max_size"><a class="link" href="#kudu-tserver_log_container_max_size"><code>--log_container_max_size</code></a></h3> <div class="paragraph"> <p>Maximum size (soft) of a log container</p> @@ -5147,6 +5537,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-tserver_scanner_max_wait_ms"><a class="link" href="#kudu-tserver_scanner_max_wait_ms"><code>--scanner_max_wait_ms</code></a></h3> +<div class="paragraph"> +<p>The maximum amount of time (in milliseconds) we’ll hang a scanner thread waiting for safe time to advance or transactions to commit, even if its deadline allows waiting longer.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-tserver_num_tablets_to_copy_simultaneously"><a class="link" href="#kudu-tserver_num_tablets_to_copy_simultaneously"><code>--num_tablets_to_copy_simultaneously</code></a></h3> +<div class="paragraph"> +<p>Number of threads available to copy tablets from remote servers.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-tserver_num_tablets_to_open_simultaneously"><a class="link" href="#kudu-tserver_num_tablets_to_open_simultaneously"><code>--num_tablets_to_open_simultaneously</code></a></h3> <div class="paragraph"> <p>Number of threads available to open tablets during startup. If this is set to 0 (the default), then the number of bootstrap threads will be set based on the number of data directories. If the data directories are on some very fast storage device such as SSD or a RAID array, it may make sense to manually tune this.</p> @@ -5277,6 +5719,32 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-tserver_file_cache_expiry_period_ms"><a class="link" href="#kudu-tserver_file_cache_expiry_period_ms"><code>--file_cache_expiry_period_ms</code></a></h3> +<div class="paragraph"> +<p>Period of time (in ms) between removing expired file cache descriptors</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>60000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-tserver_disable_core_dumps"><a class="link" href="#kudu-tserver_disable_core_dumps"><code>--disable_core_dumps</code></a></h3> <div class="paragraph"> <p>Disable core dumps when this process crashes.</p> @@ -5303,6 +5771,58 @@ for informational purposes only. They are subject to change or be removed withou </table> </div> <div class="sect2"> +<h3 id="kudu-tserver_log_redact_user_data"><a class="link" href="#kudu-tserver_log_redact_user_data"><code>--log_redact_user_data</code></a></h3> +<div class="paragraph"> +<p>Whether log and error messages will have row data redacted.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> +<h3 id="kudu-tserver_max_log_files"><a class="link" href="#kudu-tserver_max_log_files"><code>--max_log_files</code></a></h3> +<div class="paragraph"> +<p>Maximum number of log files to retain per severity level. The most recent log files are retained. If set to 0, all log files are retained.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect2"> <h3 id="kudu-tserver_log_target_replay_size_mb"><a class="link" href="#kudu-tserver_log_target_replay_size_mb"><code>--log_target_replay_size_mb</code></a></h3> <div class="paragraph"> <p>The target maximum size of logs to be replayed at startup. If a tablet has in-memory operations that are causing more than this size of logs to be retained, then the maintenance manager will prioritize flushing these operations to disk.</p> @@ -5681,6 +6201,10 @@ for informational purposes only. They are subject to change or be removed withou </li> <li> + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> + <a href="export_control.html">Export Control Notice</a> </li> </ul> @@ -5690,7 +6214,7 @@ for informational purposes only. They are subject to change or be removed withou </div> <footer class="footer"> <p class="small"> - Copyright © 2016 The Apache Software Foundation. Last updated 2016-11-21 07:26:08 PST + Copyright © 2016 The Apache Software Foundation. Last updated 2017-01-18 10:16:01 PST </p> </footer> </div> http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/contributing.html ---------------------------------------------------------------------- diff --git a/docs/contributing.html b/docs/contributing.html index a1daeee..694c587 100644 --- a/docs/contributing.html +++ b/docs/contributing.html @@ -350,12 +350,8 @@ keep under 80 where possible, but you can spill over to 100 or so if necessary.< <div class="paragraph"> <div class="title">Smart Pointers and Singly-Owned Pointers</div> <p>Generally, most objects should have clear "single-owner" semantics. -Most of the time, singly-owned objects can be wrapped in a <code>gscoped_ptr<></code> -or in a <code>unique_ptr</code> which ensures deletion on scope -exit and prevents accidental copying. -<code>gscoped_ptr</code> is similar to C++11’s <code>unique_ptr</code> in that it has a <code>release</code> -method and also provides emulated <code>move</code> semantics (see <em>gscoped_ptr.h</em> for -example usage).</p> +Most of the time, singly-owned objects can be wrapped in a <code>unique_ptr<></code> +which ensures deletion on scope exit and prevents accidental copying.</p> </div> <div class="paragraph"> <p>If an object is singly owned, but referenced from multiple places, such as when @@ -371,12 +367,21 @@ as in the following example.</p> }</code></pre> </div> </div> -<div class="paragraph"> -<p>If you use raw pointers within STL collections or inside of vectors and other containers, -associate a comment with the container, which explains the ownership -semantics (owned or un-owned). Use utility code from <em>gutil/stl_util.h</em>, such as -<code>STLDeleteElements</code> or <code>ElementDeleter</code>, to ease handling of deletion of the -contained elements.</p> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +Older parts of the Kudu code base use <code>gscoped_ptr</code> instead of +<code>unique_ptr</code>. These are hold-overs from before Kudu adopted C++11. +New code should not use <code>gscoped_ptr</code> except when necessary to interface +with existing code. Alternatively, consider updating usages as you come +across them. +</td> +</tr> +</table> </div> <div class="admonitionblock warning"> <table> @@ -785,6 +790,10 @@ to the official Kudu documentation.</p> </li> <li> + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> + <a href="export_control.html">Export Control Notice</a> </li> </ul> @@ -794,7 +803,7 @@ to the official Kudu documentation.</p> </div> <footer class="footer"> <p class="small"> - Copyright © 2016 The Apache Software Foundation. Last updated 2016-11-17 10:36:43 PST + Copyright © 2016 The Apache Software Foundation. Last updated 2017-01-12 20:05:28 PST </p> </footer> </div> http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/developing.html ---------------------------------------------------------------------- diff --git a/docs/developing.html b/docs/developing.html index aeee513..ed42db3 100644 --- a/docs/developing.html +++ b/docs/developing.html @@ -220,36 +220,22 @@ See <a href="quickstart.html">Quickstart</a> for more information.</p> <div class="sect2"> <h3 id="_maven_artifacts"><a class="link" href="#_maven_artifacts">Maven Artifacts</a></h3> <div class="paragraph"> -<p>The following Maven <code><dependency></code> element is valid for the Kudu public beta:</p> +<p>The following Maven <code><dependency></code> element is valid for the Apache Kudu public release +(since 1.0.0):</p> </div> <div class="listingblock"> <div class="content"> <pre class="highlight"><code class="language-xml" data-lang="xml"><dependency> <groupId>org.apache.kudu</groupId> <artifactId>kudu-client</artifactId> - <version>0.5.0</version> + <version>1.1.0</version> </dependency></code></pre> </div> </div> <div class="paragraph"> -<p>Because the Maven artifacts are not in Maven Central, use the following <code><repository></code> -element:</p> -</div> -<div class="listingblock"> -<div class="content"> -<pre class="highlight"><code class="language-xml" data-lang="xml"><repository> - <id>cdh.repo</id> - <name>Cloudera Repositories</name> - <url>https://repository.cloudera.com/artifactory/cloudera-repos</url> - <snapshots> - <enabled>false</enabled> - </snapshots> -</repository></code></pre> -</div> -</div> -<div class="paragraph"> -<p>See subdirectories of <a href="https://github.com/cloudera/kudu-examples/tree/master/java"; class="bare">https://github.com/cloudera/kudu-examples/tree/master/java</a> for -example Maven pom.xml files.</p> +<p>Convenience binary artifacts for the Java client and various Java integrations (e.g. Spark, Flume) +are also now available via the <a href="http://repository.apache.org";>ASF Maven repository</a> and +<a href="https://mvnrepository.com/artifact/org.apache.kudu";>Maven Central repository</a>.</p> </div> </div> </div> @@ -267,12 +253,23 @@ and using Impala with Kudu, including several <code>impala-shell</code> examples <h2 id="_kudu_integration_with_spark"><a class="link" href="#_kudu_integration_with_spark">Kudu Integration with Spark</a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Kudu integrates with Spark through the Data Source API as of version 0.9. -Include the kudu-spark jar using the --jars option:</p> +<p>Kudu integrates with Spark through the Data Source API as of version 1.0.0. +Include the kudu-spark dependency using the --packages option:</p> +</div> +<div class="paragraph"> +<p>Use the kudu-spark_2.10 artifact if using Spark with Scala 2.10</p> </div> <div class="listingblock"> <div class="content"> -<pre class="highlight"><code>spark-shell --jars kudu-spark-0.9.0.jar</code></pre> +<pre class="highlight"><code>spark-shell --packages org.apache.kudu:kudu-spark_2.10:1.1.0</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Use kudu-spark2_2.11 artifact if using Spark 2 with Scala 2.11</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>spark-shell --packages org.apache.kudu:kudu-spark2_2.11:1.1.0</code></pre> </div> </div> <div class="paragraph"> @@ -325,7 +322,34 @@ kuduContext.deleteTable("unwanted_table")</code></pre> </div> </div> <div class="sect2"> -<h3 id="_kudu_python_client"><a class="link" href="#_kudu_python_client">Kudu Python Client</a></h3> +<h3 id="_spark_integration_known_issues_and_limitations"><a class="link" href="#_spark_integration_known_issues_and_limitations">Spark Integration Known Issues and Limitations</a></h3> +<div class="ulist"> +<ul> +<li> +<p>Kudu tables with a name containing upper case or non-ascii characters must be +assigned an alternate name when registered as a temporary table.</p> +</li> +<li> +<p>Kudu tables with a column name containing upper case or non-ascii characters +may not be used with SparkSQL. Non-primary key columns may be renamed in Kudu +to work around this issue.</p> +</li> +<li> +<p><code>NULL</code>, <code>NOT NULL</code>, <code><></code>, <code>OR</code>, <code>LIKE</code>, and <code>IN</code> predicates are not pushed to +Kudu, and instead will be evaluated by the Spark task.</p> +</li> +<li> +<p>Kudu does not support all types supported by Spark SQL, such as <code>Date</code>, +<code>Decimal</code> and complex types.</p> +</li> +</ul> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_kudu_python_client"><a class="link" href="#_kudu_python_client">Kudu Python Client</a></h2> +<div class="sectionbody"> <div class="paragraph"> <p>The Kudu Python client provides a Python friendly interface to the C++ client API. The sample below demonstrates the use of part of the Python client.</p> @@ -389,35 +413,6 @@ result = scanner.open().read_all_tuples()</code></pre> </div> </div> </div> -<div class="sect2"> -<h3 id="_spark_integration_known_issues_and_limitations"><a class="link" href="#_spark_integration_known_issues_and_limitations">Spark Integration Known Issues and Limitations</a></h3> -<div class="ulist"> -<ul> -<li> -<p>The Kudu Spark integration is tested and developed against Spark 1.6 and Scala -2.10.</p> -</li> -<li> -<p>Kudu tables with a name containing upper case or non-ascii characters must be -assigned an alternate name when registered as a temporary table.</p> -</li> -<li> -<p>Kudu tables with a column name containing upper case or non-ascii characters -may not be used with SparkSQL. Non-primary key columns may be renamed in Kudu -to work around this issue.</p> -</li> -<li> -<p><code>NULL</code>, <code>NOT NULL</code>, <code><></code>, <code>OR</code>, <code>LIKE</code>, and <code>IN</code> predicates are not pushed to -Kudu, and instead will be evaluated by the Spark task.</p> -</li> -<li> -<p>Kudu does not support all types supported by Spark SQL, such as <code>Date</code>, -<code>Decimal</code> and complex types.</p> -</li> -</ul> -</div> -</div> -</div> </div> <div class="sect1"> <h2 id="_integration_with_mapreduce_yarn_and_other_frameworks"><a class="link" href="#_integration_with_mapreduce_yarn_and_other_frameworks">Integration with MapReduce, YARN, and Other Frameworks</a></h2> @@ -483,10 +478,10 @@ using YARN and Spark in the future.</p> <li><a href="#_example_impala_commands_with_kudu">Example Impala Commands With Kudu</a></li> <li><a href="#_kudu_integration_with_spark">Kudu Integration with Spark</a> <ul class="sectlevel2"> -<li><a href="#_kudu_python_client">Kudu Python Client</a></li> <li><a href="#_spark_integration_known_issues_and_limitations">Spark Integration Known Issues and Limitations</a></li> </ul> </li> +<li><a href="#_kudu_python_client">Kudu Python Client</a></li> <li><a href="#_integration_with_mapreduce_yarn_and_other_frameworks">Integration with MapReduce, YARN, and Other Frameworks</a></li> </ul> </li> @@ -512,6 +507,10 @@ using YARN and Spark in the future.</p> </li> <li> + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> + <a href="export_control.html">Export Control Notice</a> </li> </ul> @@ -521,7 +520,7 @@ using YARN and Spark in the future.</p> </div> <footer class="footer"> <p class="small"> - Copyright © 2016 The Apache Software Foundation. Last updated 2016-11-14 22:22:26 PST + Copyright © 2016 The Apache Software Foundation. Last updated 2017-01-12 20:05:28 PST </p> </footer> </div> http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/export_control.html ---------------------------------------------------------------------- diff --git a/docs/export_control.html b/docs/export_control.html index d61f637..6af62db 100644 --- a/docs/export_control.html +++ b/docs/export_control.html @@ -222,6 +222,10 @@ to provide secure authentication and TLS-protected transport.</p> <a href="configuration_reference.html">Kudu Configuration Reference</a> </li> <li> + + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> <span class="active-toc">Export Control Notice</span> </li> http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/index.html ---------------------------------------------------------------------- diff --git a/docs/index.html b/docs/index.html index 7ccacba..a56ba05 100644 --- a/docs/index.html +++ b/docs/index.html @@ -118,7 +118,7 @@ limitations under the License. <div id="preamble"> <div class="sectionbody"> <div class="paragraph"> -<p>Kudu is a columnar storage manager developed for the Hadoop platform. Kudu shares +<p>Kudu is a columnar storage manager developed for the Apache Hadoop platform. Kudu shares the common technical properties of Hadoop ecosystem applications: it runs on commodity hardware, is horizontally scalable, and supports highly available operation.</p> </div> @@ -134,8 +134,8 @@ hardware, is horizontally scalable, and supports highly available operation.</p> <p>Integration with MapReduce, Spark and other Hadoop ecosystem components.</p> </li> <li> -<p>Tight integration with Cloudera Impala, making it a good, mutable alternative -to using HDFS with Parquet.</p> +<p>Tight integration with Apache Impala (incubating), making it a good, mutable +alternative to using HDFS with Apache Parquet.</p> </li> <li> <p>Strong but flexible consistency model, allowing you to choose consistency @@ -202,9 +202,9 @@ refreshes of the predictive model based on all historic data</p> <div class="sectionbody"> <div class="dlist"> <dl> -<dt class="hdlist1"><code>CREATE TABLE</code></dt> +<dt class="hdlist1"><code>CREATE/ALTER/DROP TABLE</code></dt> <dd> -<p>Impala supports creating and dropping tables using Kudu as the persistence layer. +<p>Impala supports creating, altering, and dropping tables using Kudu as the persistence layer. The tables follow the same internal / external approach as other tables in Impala, allowing for flexible data ingestion and querying.</p> </dd> @@ -218,8 +218,7 @@ any other Impala table like those using HDFS or HBase for persistence.</p> <p>Impala supports the <code>UPDATE</code> and <code>DELETE</code> SQL commands to modify existing data in a Kudu table row-by-row or as a batch. The syntax of the SQL commands is chosen to be as compatible as possible with existing standards. In addition to simple <code>DELETE</code> -or <code>UPDATE</code> commands, you can specify complex joins with a <code>FROM</code> clause in a subquery. -Not all types of joins have been tested.</p> +or <code>UPDATE</code> commands, you can specify complex joins with a <code>FROM</code> clause in a subquery.</p> </dd> <dt class="hdlist1">Flexible Partitioning</dt> <dd> @@ -242,6 +241,10 @@ to Parquet in many workloads.</p> </dd> </dl> </div> +<div class="paragraph"> +<p>For more details regarding querying data stored in Kudu using Impala, please +refer to the Impala documentation.</p> +</div> </div> </div> <div class="sect1"> @@ -280,15 +283,16 @@ a totally ordered primary key. A table is split into segments called tablets.</p </div> <div class="paragraph"> <div class="title">Tablet</div> -<p>A <em>tablet</em> is a contiguous segment of a table. A given tablet is -replicated on multiple tablet servers, and one of these replicas is considered -the leader tablet. Any replica can service reads, and writes require consensus -among the set of tablet servers serving the tablet.</p> +<p>A <em>tablet</em> is a contiguous segment of a table, similar to a <em>partition</em> in +other data storage engines or relational databases. A given tablet is +replicated on multiple tablet servers, and at any given point in time, +one of these replicas is considered the leader tablet. Any replica can service +reads, and writes require consensus among the set of tablet servers serving the tablet.</p> </div> <div class="paragraph"> <div class="title">Tablet Server</div> <p>A <em>tablet server</em> stores and serves tablets to clients. For a -given tablet, one tablet server serves the lead tablet, and the others serve +given tablet, one tablet server acts as a leader, and the others act as follower replicas of that tablet. Only leaders service write requests, while leaders or followers each service read requests. Leaders are elected using <a href="#raft">Raft Consensus Algorithm</a>. One tablet server can serve multiple tablets, and one tablet can be served @@ -303,7 +307,7 @@ disappears, a new master is elected using <a href="#raft">Raft Consensus Algorit </div> <div class="paragraph"> <p>The master also coordinates metadata operations for clients. For example, when -creating a new table, the client internally sends an RPC to the master. The +creating a new table, the client internally sends the request to the master. The master writes the metadata for the new table into the catalog table, and coordinates the process of creating tablets on the tablet servers.</p> </div> @@ -328,7 +332,11 @@ in a majority of replicas it is acknowledged to the client. A given group of <co <div class="title">Catalog Table</div> <p>The <em>catalog table</em> is the central location for metadata of Kudu. It stores information about tables and tablets. The catalog -table is accessible to clients via the master, using the client API.</p> +table may not be read or written directly. Instead, it is accessible +only via metadata operations exposed in the client API.</p> +</div> +<div class="paragraph"> +<p>The catalog table stores two categories of metadata:</p> </div> <div class="dlist"> <dl> @@ -531,6 +539,10 @@ formats using Impala, without the need to change your legacy systems.</p> </li> <li> + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> + <a href="export_control.html">Export Control Notice</a> </li> </ul> @@ -540,7 +552,7 @@ formats using Impala, without the need to change your legacy systems.</p> </div> <footer class="footer"> <p class="small"> - Copyright © 2016 The Apache Software Foundation. Last updated 2016-10-25 14:39:46 PDT + Copyright © 2016 The Apache Software Foundation. Last updated 2017-01-12 20:05:28 PST </p> </footer> </div> http://git-wip-us.apache.org/repos/asf/kudu-site/blob/9b792926/docs/installation.html ---------------------------------------------------------------------- diff --git a/docs/installation.html b/docs/installation.html index 7706736..1e12efc 100644 --- a/docs/installation.html +++ b/docs/installation.html @@ -525,7 +525,7 @@ on a version older than 7.0, the Red Hat Developer Toolset must be installed <div class="content"> <pre>$ sudo yum install autoconf automake cyrus-sasl-devel cyrus-sasl-gssapi \ cyrus-sasl-plain gcc gcc-c++ gdb git krb5-server krb5-workstation libtool \ - make openssl-devel patch pkgconfig redhat-lsb-core rsync unzip vim-common</pre> + make openssl-devel patch pkgconfig redhat-lsb-core rsync unzip vim-common which</pre> </div> </div> </li> @@ -534,8 +534,8 @@ on a version older than 7.0, the Red Hat Developer Toolset must be installed Toolset.</p> <div class="listingblock"> <div class="content"> -<pre>$ DTLS_RPM=rhscl-devtoolset-3-epel-6-x86_64.noarch.rpm -$ DTLS_RPM_URL=https://www.softwarecollections.org/en/scls/rhscl/devtoolset-3/epel-6-x86_64/download/${DTLS_RPM} +<pre>$ DTLS_RPM=rhscl-devtoolset-3-epel-6-x86_64-1-2.noarch.rpm +$ DTLS_RPM_URL=https://www.softwarecollections.org/repos/rhscl/devtoolset-3/epel-6-x86_64/noarch/${DTLS_RPM} $ wget ${DTLS_RPM_URL} -O ${DTLS_RPM} $ sudo yum install -y scl-utils ${DTLS_RPM} $ sudo yum install -y devtoolset-3-toolchain</pre> @@ -620,9 +620,9 @@ automated deployment scenario. It skips the steps marked <strong>Optional</stron sudo yum -y install autoconf automake cyrus-sasl-devel cyrus-sasl-gssapi \ cyrus-sasl-plain gcc gcc-c++ gdb git krb5-server krb5-workstation libtool \ - make openssl-devel patch pkgconfig redhat-lsb-core rsync unzip vim-common -DTLS_RPM=rhscl-devtoolset-3-epel-6-x86_64.noarch.rpm -DTLS_RPM_URL=https://www.softwarecollections.org/en/scls/rhscl/devtoolset-3/epel-6-x86_64/download/${DTLS_RPM} + make openssl-devel patch pkgconfig redhat-lsb-core rsync unzip vim-common which +DTLS_RPM=rhscl-devtoolset-3-epel-6-x86_64-1-2.noarch.rpm +DTLS_RPM_URL=https://www.softwarecollections.org/repos/rhscl/devtoolset-3/epel-6-x86_64/noarch/${DTLS_RPM} wget ${DTLS_RPM_URL} -O ${DTLS_RPM} sudo yum install -y scl-utils ${DTLS_RPM} sudo yum install -y devtoolset-3-toolchain @@ -650,7 +650,7 @@ make -j4</code></pre> <div class="listingblock"> <div class="content"> <pre>$ sudo apt-get install autoconf automake curl g++ gcc gdb git \ - krb5-admin-server krb5-kdc krb5-user libsasl2-dev libsasl2-modules \ + krb5-admin-server krb5-kdc krb5-user libkrb5-dev libsasl2-dev libsasl2-modules \ libsasl2-modules-gssapi-mit libssl-dev libtool lsb-release make ntp openssl \ patch pkg-config python rsync unzip vim-common</pre> </div> @@ -660,7 +660,7 @@ make -j4</code></pre> <p>Optional: Install additional packages to build the documentation</p> <div class="listingblock"> <div class="content"> -<pre>$ sudo apt-get install xsltproc zlib1g-dev</pre> +<pre>$ sudo apt-get install xsltproc zlib1g-dev gem ruby-dev</pre> </div> </div> </li> @@ -728,7 +728,7 @@ the steps marked <strong>Optional</strong> above.</p> <pre class="highlight"><code class="language-bash" data-lang="bash">#!/bin/bash sudo apt-get -y install autoconf automake curl g++ gcc gdb git \ - krb5-admin-server krb5-kdc krb5-user libsasl2-dev libsasl2-modules \ + krb5-admin-server krb5-kdc krb5-user libkrb5-dev libsasl2-dev libsasl2-modules \ libsasl2-modules-gssapi-mit libssl-dev libtool lsb-release make ntp openssl \ patch pkg-config python rsync unzip vim-common git clone https://github.com/apache/kudu @@ -754,7 +754,7 @@ make -j4</code></pre> <div class="listingblock"> <div class="content"> <pre>$ sudo zypper install autoconf automake curl cyrus-sasl-devel \ - cyrus-sasl-gssapi gcc gcc-c++ gdb git libtool lsb-release make ntp \ + cyrus-sasl-gssapi gcc gcc-c++ gdb git krb5-devel libtool lsb-release make ntp \ openssl-devel patch pkg-config python rsync unzip vim</pre> </div> </div> @@ -816,7 +816,7 @@ the steps marked <strong>Optional</strong> above.</p> <pre class="highlight"><code class="language-bash" data-lang="bash">#!/bin/bash sudo zypper install -y autoconf automake curl cyrus-sasl-devel \ - cyrus-sasl-gssapi gcc gcc-c++ gdb git libtool lsb-release make ntp \ + cyrus-sasl-gssapi gcc gcc-c++ gdb git krb5-devel libtool lsb-release make ntp \ openssl-devel patch pkg-config python rsync unzip vim git clone https://github.com/apache/kudu cd kudu @@ -862,7 +862,8 @@ for more information.</p> <p>Install the prerequisite libraries, if they are not installed.</p> <div class="listingblock"> <div class="content"> -<pre>$ brew install autoconf automake cmake krb5 libtool openssl pkg-config pstree</pre> +<pre>$ brew tap homebrew/dupes +$ brew install autoconf automake cmake krb5 libtool openssl pkg-config pstree</pre> </div> </div> </li> @@ -938,6 +939,7 @@ are installed.</p> <div class="content"> <pre>#!/bin/bash +brew tap homebrew/dupes brew install autoconf automake cmake krb5 libtool openssl pkg-config pstree git clone https://github.com/apache/kudu cd kudu @@ -1244,6 +1246,10 @@ $ sudo service kudu-tserver start</code></pre> </li> <li> + <a href="known_issues.html">Known Issues and Limitations</a> + </li> + <li> + <a href="export_control.html">Export Control Notice</a> </li> </ul> @@ -1253,7 +1259,7 @@ $ sudo service kudu-tserver start</code></pre> </div> <footer class="footer"> <p class="small"> - Copyright © 2016 The Apache Software Foundation. Last updated 2016-11-14 15:52:59 PST + Copyright © 2016 The Apache Software Foundation. Last updated 2017-01-12 12:48:06 PST </p> </footer> </div>
