http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/kudu-tserver_configuration_reference_unsupported.html ---------------------------------------------------------------------- diff --git a/docs/kudu-tserver_configuration_reference_unsupported.html b/docs/kudu-tserver_configuration_reference_unsupported.html index 036bbab..e82e3a7 100644 --- a/docs/kudu-tserver_configuration_reference_unsupported.html +++ b/docs/kudu-tserver_configuration_reference_unsupported.html @@ -2,7 +2,7 @@ title: kudu-tserver Unsupported Flags layout: default active_nav: docs -last_updated: 'Last updated 2017-02-02 14:03:11 PST' +last_updated: 'Last updated 2017-03-20 16:43:12 PDT' --- <!-- @@ -62,6 +62,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_min_compression_ratio"><a class="link" href="#kudu-tserver_min_compression_ratio"><code>--min_compression_ratio</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>If a column compression codec is configured, but the codec is unable to achieve a compression ratio at least as good as the configured value, then the data will be written uncompressed. This will reduce CPU overhead on the read side at the expense of a small amount of extra space if the codec encounters portions of data that are not easily compressible.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">double</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>0.90000000000000002</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_cfile_default_block_size"><a class="link" href="#kudu-tserver_cfile_default_block_size"><code>--cfile_default_block_size</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -342,6 +370,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_log_compression_codec"><a class="link" href="#kudu-tserver_log_compression_codec"><code>--log_compression_codec</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Codec to use for compressing WAL segments.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>LZ4</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_log_max_segments_to_retain"><a class="link" href="#kudu-tserver_log_max_segments_to_retain"><code>--log_max_segments_to_retain</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -849,7 +905,7 @@ for informational purposes only. They are subject to change or be removed withou <h2 id="kudu-tserver_block_manager_max_open_files"><a class="link" href="#kudu-tserver_block_manager_max_open_files"><code>--block_manager_max_open_files</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Maximum number of open file descriptors to be used for data blocks. If 0, there is no limit. If -1, Kudu will use half of its resource limit as per getrlimit(). This is a soft limit.</p> +<p>Maximum number of open file descriptors to be used for data blocks. If 0, there is no limit. If -1, Kudu will use 40% of its resource limit as per getrlimit(). This is a soft limit.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1070,10 +1126,10 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_rpc_default_keepalive_time_ms"><a class="link" href="#kudu-tserver_rpc_default_keepalive_time_ms"><code>--rpc_default_keepalive_time_ms</code></a></h2> +<h2 id="kudu-tserver_rpc_ca_certificate_file"><a class="link" href="#kudu-tserver_rpc_ca_certificate_file"><code>--rpc_ca_certificate_file</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>If an RPC connection from a client is idle for this amount of time, the server will disconnect the client.</p> +<p>Path to the PEM encoded X509 certificate of the trusted external certificate authority. The provided certificate should be the root issuer of the certificate passed in '--rpc_certificate_file'.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1083,25 +1139,25 @@ for informational purposes only. They are subject to change or be removed withou <tbody> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>65000</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">none</p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> </tr> </tbody> </table> </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_rpc_ssl_certificate_authority"><a class="link" href="#kudu-tserver_rpc_ssl_certificate_authority"><code>--rpc_ssl_certificate_authority</code></a></h2> +<h2 id="kudu-tserver_rpc_certificate_file"><a class="link" href="#kudu-tserver_rpc_certificate_file"><code>--rpc_certificate_file</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Path to the certificate authority to be used by the client side of the connection to verify the validity of the certificate presented by the server.</p> +<p>Path to a PEM encoded X509 certificate to use for securing RPC connections with SSL/TLS. If set, '--rpc_private_key_file' and '--rpc_ca_certificate_file' must be set as well.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1126,10 +1182,10 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_rpc_ssl_private_key"><a class="link" href="#kudu-tserver_rpc_ssl_private_key"><code>--rpc_ssl_private_key</code></a></h2> +<h2 id="kudu-tserver_rpc_default_keepalive_time_ms"><a class="link" href="#kudu-tserver_rpc_default_keepalive_time_ms"><code>--rpc_default_keepalive_time_ms</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Path to the private key to be used to complement the public key present in --ssl_server_certificate</p> +<p>If an RPC connection from a client is idle for this amount of time, the server will disconnect the client.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1139,25 +1195,25 @@ for informational purposes only. They are subject to change or be removed withou <tbody> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">none</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>65000</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> </tr> </tbody> </table> </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_rpc_ssl_server_certificate"><a class="link" href="#kudu-tserver_rpc_ssl_server_certificate"><code>--rpc_ssl_server_certificate</code></a></h2> +<h2 id="kudu-tserver_rpc_private_key_file"><a class="link" href="#kudu-tserver_rpc_private_key_file"><code>--rpc_private_key_file</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Path to the SSL certificate to be used for the RPC layer.</p> +<p>Path to a PEM encoded private key paired with the certificate from '--rpc_certificate_file'</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1182,10 +1238,10 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_server_require_kerberos"><a class="link" href="#kudu-tserver_server_require_kerberos"><code>--server_require_kerberos</code></a></h2> +<h2 id="kudu-tserver_rpc_encrypt_loopback_connections"><a class="link" href="#kudu-tserver_rpc_encrypt_loopback_connections"><code>--rpc_encrypt_loopback_connections</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Whether to force all inbound RPC connections to authenticate with Kerberos</p> +<p>Whether to encrypt data transfer on RPC connections that stay within a single host. Encryption here is likely to offer no additional security benefit since only a local 'root' user could intercept the traffic, and wire encryption does not suitably protect against such an attacker.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1203,7 +1259,7 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> </tr> </tbody> </table> @@ -1406,7 +1462,7 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_kerberos_principal"><a class="link" href="#kudu-tserver_kerberos_principal"><code>--kerberos_principal</code></a></h2> +<h2 id="kudu-tserver_principal"><a class="link" href="#kudu-tserver_principal"><code>--principal</code></a></h2> <div class="sectionbody"> <div class="paragraph"> <p>Kerberos principal that this daemon will log in as. The special token _HOST will be replaced with the FQDN of the local host.</p> @@ -1427,6 +1483,34 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">unsafe,experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> +<h2 id="kudu-tserver_ipki_server_key_size"><a class="link" href="#kudu-tserver_ipki_server_key_size"><code>--ipki_server_key_size</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>the number of bits for server cert’s private key. The server cert is used for TLS connections to and from clients and other servers.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>2048</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> <td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> </tr> </tbody> @@ -1434,10 +1518,10 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_keytab"><a class="link" href="#kudu-tserver_keytab"><code>--keytab</code></a></h2> +<h2 id="kudu-tserver_rpc_tls_ciphers"><a class="link" href="#kudu-tserver_rpc_tls_ciphers"><code>--rpc_tls_ciphers</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Path to the Kerberos Keytab for this server</p> +<p>The cipher suite preferences to use for TLS-secured RPC connections. Uses the OpenSSL cipher preference list format. See man (1) ciphers for more information.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -1451,11 +1535,39 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">none</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> +<h2 id="kudu-tserver_rpc_tls_min_protocol"><a class="link" href="#kudu-tserver_rpc_tls_min_protocol"><code>--rpc_tls_min_protocol</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The minimum protocol version to allow when for securing RPC connections with TLS. May be one of 'TLSv1', 'TLSv1.1', or 'TLSv1.2'.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLSv1</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> </tr> </tbody> </table> @@ -1686,6 +1798,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_webserver_enabled"><a class="link" href="#kudu-tserver_webserver_enabled"><code>--webserver_enabled</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Whether to enable the web server on this daemon. NOTE: disabling the web server is also likely to prevent monitoring systems from properly capturing metrics.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_metrics_log_interval_ms"><a class="link" href="#kudu-tserver_metrics_log_interval_ms"><code>--metrics_log_interval_ms</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -1742,6 +1882,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_webserver_x_frame_options"><a class="link" href="#kudu-tserver_webserver_x_frame_options"><code>--webserver_x_frame_options</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The webserver will add an 'X-Frame-Options' HTTP header with this value to all responses. This can help prevent clickjacking attacks.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>DENY</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_webserver_doc_root"><a class="link" href="#kudu-tserver_webserver_doc_root"><code>--webserver_doc_root</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -1938,6 +2106,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_deltafile_default_compression_codec"><a class="link" href="#kudu-tserver_deltafile_default_compression_codec"><code>--deltafile_default_compression_codec</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The compression codec used when writing deltafiles.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>lz4</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_default_composite_key_index_block_size_bytes"><a class="link" href="#kudu-tserver_default_composite_key_index_block_size_bytes"><code>--default_composite_key_index_block_size_bytes</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -2218,6 +2414,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_undo_delta_block_gc_init_budget_millis"><a class="link" href="#kudu-tserver_undo_delta_block_gc_init_budget_millis"><code>--undo_delta_block_gc_init_budget_millis</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The maximum number of milliseconds we will spend initializing UNDO delta blocks per invocation of UndoDeltaBlockGCOp. Existing delta blocks must be initialized once per process startup to determine when they can be deleted.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1000</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced,evolving</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_flush_threshold_mb"><a class="link" href="#kudu-tserver_flush_threshold_mb"><code>--flush_threshold_mb</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -2246,6 +2470,34 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_flush_threshold_secs"><a class="link" href="#kudu-tserver_flush_threshold_secs"><code>--flush_threshold_secs</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Number of seconds after which a non-empty MemRowSet will become flushable even if it is not large.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>120</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_tablet_transaction_memory_limit_mb"><a class="link" href="#kudu-tserver_tablet_transaction_memory_limit_mb"><code>--tablet_transaction_memory_limit_mb</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -2582,10 +2834,10 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_never_fsync"><a class="link" href="#kudu-tserver_never_fsync"><code>--never_fsync</code></a></h2> +<h2 id="kudu-tserver_env_use_fsync"><a class="link" href="#kudu-tserver_env_use_fsync"><code>--env_use_fsync</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Never fsync() anything to disk. This is used by certain test cases to speed up runtime. This is very unsafe to use in production.</p> +<p>Use fsync(2) instead of fdatasync(2) for synchronizing dirty data to disk.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -2603,17 +2855,17 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">unsafe,advanced</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> </tr> </tbody> </table> </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_suicide_on_eio"><a class="link" href="#kudu-tserver_suicide_on_eio"><code>--suicide_on_eio</code></a></h2> +<h2 id="kudu-tserver_never_fsync"><a class="link" href="#kudu-tserver_never_fsync"><code>--never_fsync</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Kill the process if an I/O operation results in EIO</p> +<p>Never fsync() anything to disk. This is used by certain test cases to speed up runtime. This is very unsafe to use in production.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -2627,21 +2879,21 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>false</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">unsafe,advanced</p></td> </tr> </tbody> </table> </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_writable_file_use_fsync"><a class="link" href="#kudu-tserver_writable_file_use_fsync"><code>--writable_file_use_fsync</code></a></h2> +<h2 id="kudu-tserver_suicide_on_eio"><a class="link" href="#kudu-tserver_suicide_on_eio"><code>--suicide_on_eio</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Use fsync(2) instead of fdatasync(2) for synchronizing dirty data to disk.</p> +<p>Kill the process if an I/O operation results in EIO</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -2655,7 +2907,7 @@ for informational purposes only. They are subject to change or be removed withou </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>false</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> @@ -2722,10 +2974,10 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> -<h2 id="kudu-tserver_log_redact_user_data"><a class="link" href="#kudu-tserver_log_redact_user_data"><code>--log_redact_user_data</code></a></h2> +<h2 id="kudu-tserver_redact"><a class="link" href="#kudu-tserver_redact"><code>--redact</code></a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Whether log and error messages will have row data redacted.</p> +<p>Comma-separated list of redactions. Supported redactions are 'flag', 'log' and 'all'. If 'flag' is specified, configuration flags which may include sensitive data will be redacted whenever server configuration is emitted. If 'log' is specified, row data will be redacted from log and error messages. If 'all' is specified, all of above will be redacted.</p> </div> <table class="tableblock frame-all grid-all" style="width: 50%;"> <colgroup> @@ -2735,15 +2987,43 @@ for informational purposes only. They are subject to change or be removed withou <tbody> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>all</code></p></td> </tr> <tr> <th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> -<td class="tableblock halign-left valign-top"><p class="tableblock">experimental,runtime</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> +<h2 id="kudu-tserver_umask"><a class="link" href="#kudu-tserver_umask"><code>--umask</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The umask that will be used when creating files and directories. Permissions of top-level data directories will also be modified at start-up to conform to the given umask. Changing this value may enable unauthorized local users to read or modify data stored by Kudu.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>077</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">advanced</p></td> </tr> </tbody> </table> @@ -2778,6 +3058,62 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_data_gc_min_size_mb"><a class="link" href="#kudu-tserver_data_gc_min_size_mb"><code>--data_gc_min_size_mb</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The (exclusive) minimum number of megabytes of ancient data on disk, per tablet, needed to prioritize deletion of that data.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int64</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>0</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> +<h2 id="kudu-tserver_data_gc_prioritization_prob"><a class="link" href="#kudu-tserver_data_gc_prioritization_prob"><code>--data_gc_prioritization_prob</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The probability that we will prioritize data GC over performance improvement operations. If set to 1.0, we will always prefer to delete old data before running performance improvement operations such as delta compaction.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">double</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>0.5</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">experimental</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_log_target_replay_size_mb"><a class="link" href="#kudu-tserver_log_target_replay_size_mb"><code>--log_target_replay_size_mb</code></a></h2> <div class="sectionbody"> <div class="paragraph"> @@ -2918,6 +3254,62 @@ for informational purposes only. They are subject to change or be removed withou </div> </div> <div class="sect1"> +<h2 id="kudu-tserver_enable_minidumps"><a class="link" href="#kudu-tserver_enable_minidumps"><code>--enable_minidumps</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Whether to enable minidump generation upon process crash or SIGUSR1. Currently only supported on Linux systems.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> +<h2 id="kudu-tserver_minidump_size_limit_hint_kb"><a class="link" href="#kudu-tserver_minidump_size_limit_hint_kb"><code>--minidump_size_limit_hint_kb</code></a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Size limit hint for minidump files in KB. If a minidump exceeds this value, then breakpad will reduce the stack memory it collects for each thread from 8KB to 2KB. However it will always include the full stack memory for the first 20 threads, including the thread that crashed.</p> +</div> +<table class="tableblock frame-all grid-all" style="width: 50%;"> +<colgroup> +<col style="width: 25%;"> +<col style="width: 75%;"> +</colgroup> +<tbody> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Type</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">int32</p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Default</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>20480</code></p></td> +</tr> +<tr> +<th class="tableblock halign-left valign-top"><p class="tableblock">Tags</p></th> +<td class="tableblock halign-left valign-top"><p class="tableblock">evolving,advanced</p></td> +</tr> +</tbody> +</table> +</div> +</div> +<div class="sect1"> <h2 id="kudu-tserver_local_ip_for_outbound_sockets"><a class="link" href="#kudu-tserver_local_ip_for_outbound_sockets"><code>--local_ip_for_outbound_sockets</code></a></h2> <div class="sectionbody"> <div class="paragraph">
http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/kudu_impala_integration.html ---------------------------------------------------------------------- diff --git a/docs/kudu_impala_integration.html b/docs/kudu_impala_integration.html index 4a79019..569deab 100644 --- a/docs/kudu_impala_integration.html +++ b/docs/kudu_impala_integration.html @@ -2,7 +2,7 @@ title: Using Apache Kudu with Apache Impala (incubating) layout: default active_nav: docs -last_updated: 'Last updated 2017-02-02 13:49:37 PST' +last_updated: 'Last updated 2017-02-03 13:18:53 PST' --- <!-- http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/prior_release_notes.html ---------------------------------------------------------------------- diff --git a/docs/prior_release_notes.html b/docs/prior_release_notes.html index 0ae3d43..775b2bd 100644 --- a/docs/prior_release_notes.html +++ b/docs/prior_release_notes.html @@ -2,7 +2,7 @@ title: Apache Kudu Prior Version Release Notes layout: default active_nav: docs -last_updated: 'Last updated 2017-01-26 16:14:09 PST' +last_updated: 'Last updated 2017-03-10 12:47:33 PST' --- <!-- @@ -49,6 +49,305 @@ for a list of known issues and limitations. </div> </div> <div class="sect1"> +<h2 id="rn_1.2.0"><a class="link" href="#rn_1.2.0">Release notes specific to 1.2.0</a></h2> +<div class="sectionbody"> + +</div> +</div> +<div class="sect1"> +<h2 id="rn_1.2.0_new_features"><a class="link" href="#rn_1.2.0_new_features">New features</a></h2> +<div class="sectionbody"> +<div class="ulist"> +<ul> +<li> +<p>Kudu clients and servers now redact user data such as cell values +from log messages, Java exception messages, and <code>Status</code> strings. +User metadata such as table names, column names, and partition +bounds are not redacted.</p> +<div class="paragraph"> +<p>Redaction is enabled by default, but may be disabled by setting the new +<code>log_redact_user_data</code> flag to <code>false</code>.</p> +</div> +</li> +<li> +<p>Kudu’s ability to provide consistency guarantees has been substantially +improved:</p> +<div class="ulist"> +<ul> +<li> +<p>Replicas now correctly track their "safe timestamp". This timestamp +is the maximum timestamp at which reads are guaranteed to be +repeatable.</p> +</li> +<li> +<p>A scan created using the <code>SCAN_AT_SNAPSHOT</code> mode will now +either wait for the requested snapshot to be "safe" at the replica +being scanned, or be re-routed to a replica where the requested +snapshot is "safe". This ensures that all such scans are repeatable.</p> +</li> +<li> +<p>Kudu Tablet Servers now properly retain historical data when a row +with a given primary key is inserted and deleted, followed by the +insertion of a new row with the same key. Previous versions of Kudu +would not retain history in such situations. This allows the server +to return correct results for snapshot scans with a timestamp in the +past, even in the presence of such "reinsertion" scenarios.</p> +</li> +<li> +<p>The Kudu clients now automatically retain the timestamp of their latest +successful read or write operation. Scans using the <code>READ_AT_SNAPSHOT</code> mode +without a client-provided timestamp automatically assign a timestamp +higher than the timestamp of their most recent write. Writes also propagate +the timestamp, ensuring that sequences of operations with causal dependencies +between them are assigned increasing timestamps. Together, these changes +allow clients to achieve read-your-writes consistency, and also ensure +that snapshot scans performed by other clients return causally-consistent +results.</p> +</li> +</ul> +</div> +</li> +<li> +<p>Kudu servers now automatically limit the number of log files. +The number of log files retained can be configured using the +<code>max_log_files</code> flag. By default, 10 log files will be retained +at each severity level.</p> +</li> +</ul> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_optimizations_and_improvements"><a class="link" href="#_optimizations_and_improvements">Optimizations and improvements</a></h2> +<div class="sectionbody"> +<div class="ulist"> +<ul> +<li> +<p>The logging in the Java and C++ clients has been substantially quieted. +Clients no longer log messages in normal operation unless there +is some kind of error.</p> +</li> +<li> +<p>The C++ client now includes a <code>KuduSession::SetErrorBufferSpace</code> +API which can limit the amount of memory used to buffer +errors from asynchronous operations.</p> +</li> +<li> +<p>The Java client now fetches tablet locations from the Kudu Master +in batches of 1000, increased from batches of 10 in prior versions. +This can substantially improve the performance of Spark and Impala +queries running against Kudu tables with large numbers of tablets.</p> +</li> +<li> +<p>Table metadata lock contention in the Kudu Master was substantially +reduced. This improves the performance of tablet location lookups on +large clusters with a high degree of concurrency.</p> +</li> +<li> +<p>Lock contention in the Kudu Tablet Server during high-concurrency +write workloads was also reduced. This can reduce CPU consumption and +improve performance when a large number of concurrent clients are writing +to a smaller number of a servers.</p> +</li> +<li> +<p>Lock contention when writing log messages has been substantially reduced. +This source of contention could cause high tail latencies on requests, +and when under high load could contribute to cluster instability +such as election storms and request timeouts.</p> +</li> +<li> +<p>The <code>BITSHUFFLE</code> column encoding has been optimized to use the <code>AVX2</code> +instruction set present on processors including Intel® Sandy Bridge +and later. Scans on <code>BITSHUFFLE</code>-encoded columns are now up to 30% faster.</p> +</li> +<li> +<p>The <code>kudu</code> tool now accepts hyphens as an alternative to underscores +when specifying actions. For example, <code>kudu local-replica copy-from-remote</code> +may be used as an alternative to <code>kudu local_replica copy_from_remote</code>.</p> +</li> +</ul> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="rn_1.2.0_fixed_issues"><a class="link" href="#rn_1.2.0_fixed_issues">Fixed Issues</a></h2> +<div class="sectionbody"> +<div class="ulist"> +<ul> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1508";>KUDU-1508</a> +Fixed a long-standing issue in which running Kudu on <code>ext4</code> file systems +could cause file system corruption.</p> +</li> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1399";>KUDU-1399</a> +Implemented an LRU cache for open files, which prevents running out of +file descriptors on long-lived Kudu clusters. By default, Kudu will +limit its file descriptor usage to half of its configured <code>ulimit</code>.</p> +</li> +<li> +<p><a href="http://gerrit.cloudera.org:8080/5192";>Gerrit #5192</a> +Fixed an issue which caused data corruption and crashes in the case that +a table had a non-composite (single-column) primary key, and that column +was specified to use <code>DICT_ENCODING</code> or <code>BITSHUFFLE</code> encodings. If a +table with an affected schema was written in previous versions of Kudu, +the corruption will not be automatically repaired; users are encouraged +to re-insert such tables after upgrading to Kudu 1.2 or later.</p> +</li> +<li> +<p><a href="http://gerrit.cloudera.org:8080/5541";>Gerrit #5541</a> +Fixed a bug in the Spark <code>KuduRDD</code> implementation which could cause +rows in the result set to be silently skipped in some cases.</p> +</li> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1551";>KUDU-1551</a> +Fixed an issue in which the tablet server would crash on restart in the +case that it had previously crashed during the process of allocating +a new WAL segment.</p> +</li> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1764";>KUDU-1764</a> +Fixed an issue where Kudu servers would leak approximately 16-32MB of disk +space for every 10GB of data written to disk. After upgrading to Kudu +1.2 or later, any disk space leaked in previous versions will be +automatically recovered on startup.</p> +</li> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1750";>KUDU-1750</a> +Fixed an issue where the API to drop a range partition would drop any +partition with a matching lower <em>or</em> upper bound, rather than any partition +with matching lower <em>and</em> upper bound.</p> +</li> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1766";>KUDU-1766</a> +Fixed an issue in the Java client where equality predicates which compared +an integer column to its maximum possible value (e.g. <code>Integer.MAX_VALUE</code>) +would return incorrect results.</p> +</li> +<li> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1780";>KUDU-1780</a> +Fixed the <code>kudu-client</code> Java artifact to properly shade classes in the +<code>com.google.thirdparty</code> namespace. The lack of proper shading in prior +releases could cause conflicts with certain versions of Google Guava.</p> +</li> +<li> +<p><a href="http://gerrit.cloudera.org:8080/5327";>Gerrit #5327</a> +Fixed shading issues in the <code>kudu-flume-sink</code> Java artifact. The sink +now expects that Hadoop dependencies are provided by Flume, and properly +shades the Kudu client’s dependencies.</p> +</li> +<li> +<p>Fixed a few issues using the Python client library from Python 3.</p> +</li> +</ul> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="rn_1.2.0_wire_compatibility"><a class="link" href="#rn_1.2.0_wire_compatibility">Wire Protocol compatibility</a></h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Kudu 1.2.0 is wire-compatible with previous versions of Kudu:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Kudu 1.2 clients may connect to servers running Kudu 1.0. If the client uses features +that are not available on the target server, an error will be returned.</p> +</li> +<li> +<p>Kudu 1.0 clients may connect to servers running Kudu 1.2 without limitations.</p> +</li> +<li> +<p>Rolling upgrade between Kudu 1.1 and Kudu 1.2 servers is believed to be possible +though has not been sufficiently tested. Users are encouraged to shut down all nodes +in the cluster, upgrade the software, and then restart the daemons on the new version.</p> +</li> +</ul> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="rn_1.2.0_incompatible_changes"><a class="link" href="#rn_1.2.0_incompatible_changes">Incompatible Changes in Kudu 1.2.0</a></h2> +<div class="sectionbody"> +<div class="ulist"> +<ul> +<li> +<p>The replication factor of tables is now limited to a maximum of 7. In addition, +it is no longer allowed to create a table with an even replication factor.</p> +</li> +<li> +<p>The <code>GROUP_VARINT</code> encoding is now deprecated. Kudu servers have never supported +this encoding, and now the client-side constant has been deprecated to match the +server’s capabilities.</p> +</li> +</ul> +</div> +<div class="sect2"> +<h3 id="_new_restrictions_on_data_schemas_and_identifiers"><a class="link" href="#_new_restrictions_on_data_schemas_and_identifiers">New Restrictions on Data, Schemas, and Identifiers</a></h3> +<div class="paragraph"> +<p>Kudu 1.2.0 introduces several new restrictions on schemas, cell size, and identifiers:</p> +</div> +<div class="dlist"> +<dl> +<dt class="hdlist1">Number of Columns</dt> +<dd> +<p>By default, Kudu will not permit the creation of tables with +more than 300 columns. We recommend schema designs that use fewer columns for best +performance.</p> +</dd> +<dt class="hdlist1">Size of Cells</dt> +<dd> +<p>No individual cell may be larger than 64KB. The cells making up a +a composite key are limited to a total of 16KB after the internal composite-key encoding +done by Kudu. Inserting rows not conforming to these limitations will result in errors +being returned to the client.</p> +</dd> +<dt class="hdlist1">Valid Identifiers</dt> +<dd> +<p>Identifiers such as column and table names are now restricted to +be valid UTF-8 strings. Additionally, a maximum length of 256 characters is enforced.</p> +</dd> +</dl> +</div> +</div> +<div class="sect2"> +<h3 id="rn_1.2.0_client_compatibility"><a class="link" href="#rn_1.2.0_client_compatibility">Client Library Compatibility</a></h3> +<div class="ulist"> +<ul> +<li> +<p>The Kudu 1.2 Java client is API- and ABI-compatible with Kudu 1.1. Applications +written against Kudu 1.1 will compile and run against the Kudu 1.2 client and +vice-versa.</p> +</li> +<li> +<p>The Kudu 1.2 C++ client is API- and ABI-forward-compatible with Kudu 1.1. +Applications written and compiled against the Kudu 1.1 client will run without +modification against the Kudu 1.2 client. Applications written and compiled +against the Kudu 1.2 client will run without modification against the Kudu 1.1 +client unless they use one of the following new APIs:</p> +<div class="ulist"> +<ul> +<li> +<p><code>kudu::DisableSaslInitialization()</code></p> +</li> +<li> +<p><code>KuduSession::SetErrorBufferSpace(…​)</code></p> +</li> +</ul> +</div> +</li> +<li> +<p>The Kudu 1.2 Python client is API-compatible with Kudu 1.1. Applications +written against Kudu 1.1 will continue to run against the Kudu 1.2 client +and vice-versa.</p> +</li> +</ul> +</div> +</div> +</div> +</div> +<div class="sect1"> <h2 id="rn_1.1.0"><a class="link" href="#rn_1.1.0">Release notes specific to 1.1.0</a></h2> <div class="sectionbody"> @@ -104,17 +403,17 @@ IN LIST pushdown is not yet complete.</p> <li> <p>The Java client now features client-side request tracing in order to help troubleshoot timeouts. Error messages are now augmented with traces that show which servers were contacted before the -timeout occured instead of just the last error. The traces also contain RPCs that were +timeout occurred instead of just the last error. The traces also contain RPCs that were required to fulfill the client’s request, such as contacting the master to discover a tablet’s location. Note that the traces are not available for successful requests and are not -programatically queryable.</p> +programmatically queryable.</p> </li> </ul> </div> </div> </div> <div class="sect1"> -<h2 id="_optimizations_and_improvements"><a class="link" href="#_optimizations_and_improvements">Optimizations and improvements</a></h2> +<h2 id="_optimizations_and_improvements_2"><a class="link" href="#_optimizations_and_improvements_2">Optimizations and improvements</a></h2> <div class="sectionbody"> <div class="ulist"> <ul> @@ -363,7 +662,7 @@ This can provide higher throughput for ingest workloads.</p> </div> </div> <div class="sect2"> -<h3 id="_optimizations_and_improvements_2"><a class="link" href="#_optimizations_and_improvements_2">Optimizations and improvements</a></h3> +<h3 id="_optimizations_and_improvements_3"><a class="link" href="#_optimizations_and_improvements_3">Optimizations and improvements</a></h3> <div class="ulist"> <ul> <li> @@ -1194,7 +1493,7 @@ real-world use.</p> <li> <p>The master now attempts to spread tablets more evenly across the cluster during table creation. This has no impact on existing tables, but will improve the speed -at which under-replicated tabletsare re-replicated after a tablet server failure.</p> +at which under-replicated tablets are re-replicated after a tablet server failure.</p> </li> <li> <p>All licensing documents have been modified to adhere to ASF guidelines.</p> http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/quickstart.html ---------------------------------------------------------------------- diff --git a/docs/quickstart.html b/docs/quickstart.html index 201c7c6..95783fd 100644 --- a/docs/quickstart.html +++ b/docs/quickstart.html @@ -2,7 +2,7 @@ title: Apache Kudu Quickstart layout: default active_nav: docs -last_updated: 'Last updated 2017-02-02 13:49:37 PST' +last_updated: 'Last updated 2017-03-01 12:43:33 PST' --- <!-- @@ -104,7 +104,7 @@ to consult the <a href="#trouble">Troubleshooting</a> section.</p> <p>To practice some typical operations with Kudu and Impala, we’ll use the <a href="https://data.sfgov.org/Transportation/Raw-AVL-GPS-data/5fk7-ivit/data";>San Francisco MTA GPS dataset</a>. This dataset contains raw location data transmitted periodically from -sensors installed on the busses in the SF MTA’s fleet.</p> +sensors installed on the buses in the SF MTA’s fleet.</p> </div> <div class="olist arabic"> <ol class="arabic"> @@ -418,6 +418,6 @@ contain references to the previous VM’s SSH credentials. Remove any refere <div id="footnotes"> <hr> <div class="footnote" id="_footnote_1"> - <a href="#_footnoteref_1">1</a>. In addition, the script will create a host-only network between host and guest and setup an enty in the <code>/etc/hosts</code> file with the name <code>quickstart.cloudera</code> and the guest’s IP address. + <a href="#_footnoteref_1">1</a>. In addition, the script will create a host-only network between host and guest and setup an entry in the <code>/etc/hosts</code> file with the name <code>quickstart.cloudera</code> and the guest’s IP address. </div> </div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/release_notes.html ---------------------------------------------------------------------- diff --git a/docs/release_notes.html b/docs/release_notes.html index d7e8b0b..5c6bf04 100644 --- a/docs/release_notes.html +++ b/docs/release_notes.html @@ -1,8 +1,8 @@ --- -title: Apache Kudu 1.2.0 Release Notes +title: Apache Kudu 1.3.0 Release Notes layout: default active_nav: docs -last_updated: 'Last updated 2017-01-26 16:14:09 PST' +last_updated: 'Last updated 2017-03-10 12:47:33 PST' --- <!-- @@ -24,65 +24,42 @@ limitations under the License. <div class="row"> <div class="col-md-9"> -<h1>Apache Kudu 1.2.0 Release Notes</h1> +<h1>Apache Kudu 1.3.0 Release Notes</h1> <div class="sect1"> -<h2 id="rn_1.2.0_new_features"><a class="link" href="#rn_1.2.0_new_features">New features</a></h2> +<h2 id="rn_1.3.0_new_features"><a class="link" href="#rn_1.3.0_new_features">New features</a></h2> <div class="sectionbody"> <div class="ulist"> <ul> <li> -<p>Kudu clients and servers now redact user data such as cell values -from log messages, Java exception messages, and <code>Status</code> strings. -User metadata such as table names, column names, and partition -bounds are not redacted.</p> -<div class="paragraph"> -<p>Redaction is enabled by default, but may be disabled by setting the new -<code>log_redact_user_data</code> flag to <code>false</code>.</p> -</div> +<p>Kudu 1.3 adds support for strong authentication based on Kerberos. This optional feature +allows users to authenticate themselves using Kerberos tickets, and also provides +mutual authentication of servers using Kerberos credentials stored in keytabs. This +feature is optional, but recommended for deployments requiring security.</p> </li> <li> -<p>Kudu’s ability to provide consistency guarantees has been substantially -improved:</p> -<div class="ulist"> -<ul> -<li> -<p>Replicas now correctly track their "safe timestamp". This timestamp -is the maximum timestamp at which reads are guaranteed to be -repeatable.</p> +<p>Kudu 1.3 adds support for encryption of data on the network using Transport Layer Security +(TLS). Kudu will now use TLS to encrypt all network traffic between clients and servers as +well as any internal traffic among servers, with the exception of traffic determined to +be within a localhost network connection. Encryption is enabled by default whenever it can +be determined that both the client and server support the feature.</p> </li> <li> -<p>A scan created using the <code>SCAN_AT_SNAPSHOT</code> mode will now -either wait for the requested snapshot to be "safe" at the replica -being scanned, or be re-routed to a replica where the requested -snapshot is "safe". This ensures that all such scans are repeatable.</p> +<p>Kudu 1.3 adds coarse-grained service-level authorization of access to the cluster. +The operator may set up lists of permitted users who may act as administrators and +as clients of the cluster. Combined with the strong authentication feature described +above, this can enable a secure environment for some use cases. Note that fine-grained +access control (e.g. table-level or column-level) is not yet supported.</p> </li> <li> -<p>Kudu Tablet Servers now properly retain historical data when a row -with a given primary key is inserted and deleted, followed by the -insertion of a new row with the same key. Previous versions of Kudu -would not retain history in such situations. This allows the server -to return correct results for snapshot scans with a timestamp in the -past, even in the presence of such "reinsertion" scenarios.</p> +<p>Kudu 1.3 adds a background task to tablet servers which removes historical versions of +data which have fallen behind the configured data retention time. This reduces disk space +usage in all workloads, but particularly in those with a higher volume of updates or +upserts.</p> </li> <li> -<p>The Kudu clients now automatically retain the timestamp of their latest -successful read or write operation. Scans using the <code>READ_AT_SNAPSHOT</code> mode -without a client-provided timestamp automatically assign a timestamp -higher than the timestamp of their most recent write. Writes also propagate -the timestamp, ensuring that sequences of operations with causal dependencies -between them are assigned increasing timestamps. Together, these changes -allow clients to achieve read-your-writes consistency, and also ensure -that snapshot scans performed by other clients return causally-consistent -results.</p> -</li> -</ul> -</div> -</li> -<li> -<p>Kudu servers now automatically limit the number of log files. -The number of log files retained can be configured using the -<code>max_log_files</code> flag. By default, 10 log files will be retained -at each severity level.</p> +<p>Kudu now incorporates Google Breakpad, a library which writes crash reports in +the case of a server crash. These reports can be found within the configured log directory, +and can be useful during bug diagnosis.</p> </li> </ul> </div> @@ -94,223 +71,225 @@ at each severity level.</p> <div class="ulist"> <ul> <li> -<p>The logging in the Java and C++ clients has been substantially quieted. -Clients no longer log messages in normal operation unless there -is some kind of error.</p> +<p>Kudu servers will now change the file permissions of data directories and contained +data files based on a new configuration flag <code>--umask</code>. As a result, after upgrading, +permissions on disk may be more restrictive than in previous versions. The new default +configuration improves data security.</p> </li> <li> -<p>The C++ client now includes a <code>KuduSession::SetErrorBufferSpace</code> -API which can limit the amount of memory used to buffer -errors from asynchronous operations.</p> +<p>Kudu’s web UI will now redact strings which may include sensitive user data. For example, +the monitoring page which shows in-progress scans no longer includes the scanner predicate +values. The tracing and RPC diagnostics endpoints no longer include contents of RPCs which +may include table data.</p> </li> <li> -<p>The Java client now fetches tablet locations from the Kudu Master -in batches of 1000, increased from batches of 10 in prior versions. -This can substantially improve the performance of Spark and Impala -queries running against Kudu tables with large numbers of tablets.</p> +<p>By default, Kudu now reserves 1% of each configured data volume as free space. If a volume +is seen to have less than 1% of disk space free, Kudu will stop writing to that volume +to avoid completely filling up the disk.</p> </li> <li> -<p>Table metadata lock contention in the Kudu Master was substantially -reduced. This improves the performance of tablet location lookups on -large clusters with a high degree of concurrency.</p> +<p>The default encoding for numeric columns (int, float, and double) has been changed +to <code>BIT_SHUFFLE</code>. The default encoding for binary and string columns has been +changed to <code>DICT_ENCODING</code>. Dictionary encoding automatically falls back to the old +default (<code>PLAIN</code>) when cardinality is too high to be effectively encoded.</p> +<div class="paragraph"> +<p>These new defaults match the default behavior of other storage mechanisms such as + Apache Parquet and are likely to perform better out of the box.</p> +</div> </li> <li> -<p>Lock contention in the Kudu Tablet Server during high-concurrency -write workloads was also reduced. This can reduce CPU consumption and -improve performance when a large number of concurrent clients are writing -to a smaller number of a servers.</p> +<p>Kudu now uses <code>LZ4</code> compression when writing its Write Ahead Log (WAL). This improves +write performance and stability for many use cases.</p> </li> <li> -<p>Lock contention when writing log messages has been substantially reduced. -This source of contention could cause high tail latencies on requests, -and when under high load could contribute to cluster instability -such as election storms and request timeouts.</p> +<p>Kudu now uses <code>LZ4</code> compression when writing delta files. This can improve both +read and write performance as well as save substantial disk usage, especially +for workloads involving a high number of updates or upserts containing compressible +data.</p> </li> <li> -<p>The <code>BITSHUFFLE</code> column encoding has been optimized to use the <code>AVX2</code> -instruction set present on processors including Intel® Sandy Bridge -and later. Scans on <code>BITSHUFFLE</code>-encoded columns are now up to 30% faster.</p> +<p>The Kudu API now supports the ability to express <code>IS NULL</code> and <code>IS NOT NULL</code> predicates +on scanners. The Spark DataSource integration will take advantage of these new +predicates when possible.</p> </li> <li> -<p>The <code>kudu</code> tool now accepts hyphens as an alternative to underscores -when specifying actions. For example, <code>kudu local-replica copy-from-remote</code> -may be used as an alternative to <code>kudu local_replica copy_from_remote</code>.</p> +<p>Both C++ and Java clients have been optimized to prune partitions more effectively +when performing scans using the <code>IN (…​)</code> predicate.</p> +</li> +<li> +<p>The exception messages produced by the Java client are now truncated to a maximum length +of 32KB.</p> </li> </ul> </div> </div> </div> <div class="sect1"> -<h2 id="rn_1.2.0_fixed_issues"><a class="link" href="#rn_1.2.0_fixed_issues">Fixed Issues</a></h2> +<h2 id="rn_1.3.0_fixed_issues"><a class="link" href="#rn_1.3.0_fixed_issues">Fixed Issues</a></h2> <div class="sectionbody"> <div class="ulist"> <ul> <li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1508";>KUDU-1508</a> -Fixed a long-standing issue in which running Kudu on <code>ext4</code> file systems -could cause file system corruption.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1893";>KUDU-1893</a> +Fixed a critical bug in which wrong results would be returned when evaluating +predicates applied to columns added using the <code>ALTER TABLE</code> operation.</p> </li> <li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1399";>KUDU-1399</a> -Implemented an LRU cache for open files, which prevents running out of -file descriptors on long-lived Kudu clusters. By default, Kudu will -limit its file descriptor usage to half of its configured <code>ulimit</code>.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1905";>KUDU-1905</a> +Fixed a crash after inserting a row sharing a primary key with a recently-deleted +row in tables where the primary key is comprised of all of the columns.</p> </li> <li> -<p><a href="http://gerrit.cloudera.org:8080/5192";>Gerrit #5192</a> -Fixed an issue which caused data corruption and crashes in the case that -a table had a non-composite (single-column) primary key, and that column -was specified to use <code>DICT_ENCODING</code> or <code>BITSHUFFLE</code> encodings. If a -table with an affected schema was written in previous versions of Kudu, -the corruption will not be automatically repaired; users are encouraged -to re-insert such tables after upgrading to Kudu 1.2 or later.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1899";>KUDU-1899</a> +Fixed a crash after inserting a row with an empty string as the single-column +primary key.</p> </li> <li> -<p><a href="http://gerrit.cloudera.org:8080/5541";>Gerrit #5541</a> -Fixed a bug in the Spark <code>KuduRDD</code> implementation which could cause -rows in the result set to be silently skipped in some cases.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1904";>KUDU-1904</a> +Fixed a potential crash when performing random reads against a column using RLE +encoding and containing long runs of NULL values.</p> </li> <li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1551";>KUDU-1551</a> -Fixed an issue in which the tablet server would crash on restart in the -case that it had previously crashed during the process of allocating -a new WAL segment.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1853";>KUDU-1853</a> +Fixed an issue where disk space could be leaked on servers which experienced an error +during the process of copying tablet data from another server.</p> </li> <li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1764";>KUDU-1764</a> -Fixed an issue where Kudu servers would leak approximately 16-32MB of disk -space for every 10GB of data written to disk. After upgrading to Kudu -1.2 or later, any disk space leaked in previous versions will be -automatically recovered on startup.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1856";>KUDU-1856</a> +Fixed an issue in which disk space could be leaked by Kudu servers storing data on +partitions using the XFS file system. Any leaked disk space will be automatically +recovered upon upgrade.</p> </li> <li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1750";>KUDU-1750</a> -Fixed an issue where the API to drop a range partition would drop any -partition with a matching lower <em>or</em> upper bound, rather than any partition -with matching lower <em>and</em> upper bound.</p> -</li> -<li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1766";>KUDU-1766</a> -Fixed an issue in the Java client where equality predicates which compared -an integer column to its maximum possible value (e.g. <code>Integer.MAX_VALUE</code>) -would return incorrect results.</p> -</li> -<li> -<p><a href="https://issues.apache.org/jira/browse/KUDU-1780";>KUDU-1780</a> -Fixed the <code>kudu-client</code> Java artifact to properly shade classes in the -<code>com.google.thirdparty</code> namespace. The lack of proper shading in prior -releases could cause conflicts with certain versions of Google Guava.</p> -</li> -<li> -<p><a href="http://gerrit.cloudera.org:8080/5327";>Gerrit #5327</a> -Fixed shading issues in the <code>kudu-flume-sink</code> Java artifact. The sink -now expects that Hadoop dependencies are provided by Flume, and properly -shades the Kudu client’s dependencies.</p> -</li> -<li> -<p>Fixed a few issues using the Python client library from Python 3.</p> +<p><a href="https://issues.apache.org/jira/browse/KUDU-1888";>KUDU-1888</a>, +<a href="https://issues.apache.org/jira/browse/KUDU-1906";>KUDU-1906</a> +Fixed multiple issues in the Java client where operation callbacks would never be +triggered, causing the client to hang.</p> </li> </ul> </div> </div> </div> <div class="sect1"> -<h2 id="rn_1.2.0_wire_compatibility"><a class="link" href="#rn_1.2.0_wire_compatibility">Wire Protocol compatibility</a></h2> +<h2 id="rn_1.3.0_wire_compatibility"><a class="link" href="#rn_1.3.0_wire_compatibility">Wire Protocol compatibility</a></h2> <div class="sectionbody"> <div class="paragraph"> -<p>Kudu 1.2.0 is wire-compatible with previous versions of Kudu:</p> +<p>Kudu 1.3.0 is wire-compatible with previous versions of Kudu:</p> </div> <div class="ulist"> <ul> <li> -<p>Kudu 1.2 clients may connect to servers running Kudu 1.0. If the client uses features +<p>Kudu 1.3 clients may connect to servers running Kudu 1.0. If the client uses features that are not available on the target server, an error will be returned.</p> </li> <li> -<p>Kudu 1.0 clients may connect to servers running Kudu 1.2 without limitations.</p> +<p>Kudu 1.0 clients may connect to servers running Kudu 1.3 with the exception of the +below-mentioned restrictions regarding secure clusters.</p> </li> <li> -<p>Rolling upgrade between Kudu 1.1 and Kudu 1.2 servers is believed to be possible +<p>Rolling upgrade between Kudu 1.2 and Kudu 1.3 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.</p> </li> </ul> </div> +<div class="paragraph"> +<p>The authentication features newly introduced in Kudu 1.3 place the following limitations +on wire compatibility with older versions:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>If a Kudu 1.3 cluster is configured with authentication or encryption set to "required", +older clients will be unable to connect.</p> +</li> +<li> +<p>If a Kudu 1.3 cluster is configured with authentication and encryption set to "optional" +or "disabled", older clients will still be able to connect.</p> +</li> +</ul> +</div> </div> </div> <div class="sect1"> -<h2 id="rn_1.2.0_incompatible_changes"><a class="link" href="#rn_1.2.0_incompatible_changes">Incompatible Changes in Kudu 1.2.0</a></h2> +<h2 id="rn_1.3.0_incompatible_changes"><a class="link" href="#rn_1.3.0_incompatible_changes">Incompatible Changes in Kudu 1.3.0</a></h2> <div class="sectionbody"> <div class="ulist"> <ul> <li> -<p>The replication factor of tables is now limited to a maximum of 7. In addition, -it is no longer allowed to create a table with an even replication factor.</p> +<p>Due to storage format changes in Kudu 1.3, downgrade from Kudu 1.3 to earlier versions +is not supported. After upgrading to Kudu 1.3, attempting to restart with an earlier +version will result in an error.</p> +</li> +<li> +<p>In order to support running MapReduce and Spark jobs on secure clusters, these +frameworks now connect to the cluster at job submission time to retrieve authentication +credentials which can later be used by the tasks to be spawned. This means that +the process submitting jobs to Kudu clusters must have direct access to that cluster.</p> </li> <li> -<p>The <code>GROUP_VARINT</code> encoding is now deprecated. Kudu servers have never supported -this encoding, and now the client-side constant has been deprecated to match the -server’s capabilities.</p> +<p>The embedded web servers in Kudu processes now specify the <code>X-Frame-Options: DENY</code> HTTP +header which prevents embedding Kudu web pages in HTML <code>iframe</code> elements.</p> </li> </ul> </div> <div class="sect2"> -<h3 id="_new_restrictions_on_data_schemas_and_identifiers"><a class="link" href="#_new_restrictions_on_data_schemas_and_identifiers">New Restrictions on Data, Schemas, and Identifiers</a></h3> -<div class="paragraph"> -<p>Kudu 1.2.0 introduces several new restrictions on schemas, cell size, and identifiers:</p> -</div> -<div class="dlist"> -<dl> -<dt class="hdlist1">Number of Columns</dt> -<dd> -<p>By default, Kudu will not permit the creation of tables with -more than 300 columns. We recommend schema designs that use fewer columns for best -performance.</p> -</dd> -<dt class="hdlist1">Size of Cells</dt> -<dd> -<p>No individual cell may be larger than 64KB. The cells making up a -a composite key are limited to a total of 16KB after the internal composite-key encoding -done by Kudu. Inserting rows not conforming to these limitations will result in errors -being returned to the client.</p> -</dd> -<dt class="hdlist1">Valid Identifiers</dt> -<dd> -<p>Identifiers such as column and table names are now restricted to -be valid UTF-8 strings. Additionally, a maximum length of 256 characters is enforced.</p> -</dd> -</dl> -</div> -</div> -<div class="sect2"> -<h3 id="rn_1.2.0_client_compatibility"><a class="link" href="#rn_1.2.0_client_compatibility">Client Library Compatibility</a></h3> +<h3 id="rn_1.3.0_client_compatibility"><a class="link" href="#rn_1.3.0_client_compatibility">Client Library Compatibility</a></h3> +<div class="ulist"> +<ul> +<li> +<p>The Kudu 1.3 Java client library is API- and ABI-compatible with Kudu 1.2. Applications +written against Kudu 1.2 will compile and run against the Kudu 1.3 client library and +vice-versa, unless one of the following newly added APIs is used:</p> <div class="ulist"> <ul> <li> -<p>The Kudu 1.2 Java client is API- and ABI-compatible with Kudu 1.1. Applications -written against Kudu 1.1 will compile and run against the Kudu 1.2 client and -vice-versa.</p> +<p><code>[Async]KuduClient.exportAuthenticationCredentials(…​)</code> (unstable API)</p> </li> <li> -<p>The Kudu 1.2 C++ client is API- and ABI-forward-compatible with Kudu 1.1. -Applications written and compiled against the Kudu 1.1 client will run without -modification against the Kudu 1.2 client. Applications written and compiled -against the Kudu 1.2 client will run without modification against the Kudu 1.1 -client unless they use one of the following new APIs:</p> +<p><code>[Async]KuduClient.importAuthenticationCredentials(…​)</code> (unstable API)</p> +</li> +<li> +<p><code>[Async]KuduClient.getMasterAddressesAsString()</code></p> +</li> +<li> +<p><code>KuduPredicate.newIsNotNullPredicate()</code></p> +</li> +<li> +<p><code>KuduPredicate.newIsNullPredicate()</code></p> +</li> +</ul> +</div> +</li> +<li> +<p>The Kudu 1.3 C++ client is API- and ABI-forward-compatible with Kudu 1.2. +Applications written and compiled against the Kudu 1.2 client library will run without +modification against the Kudu 1.3 client library. Applications written and compiled +against the Kudu 1.3 client library will run without modification against the Kudu 1.2 +client library unless they use one of the following new APIs:</p> <div class="ulist"> <ul> <li> -<p><code>kudu::DisableSaslInitialization()</code></p> +<p><code>kudu::DisableOpenSSLInitialization()</code></p> +</li> +<li> +<p><code>KuduClientBuilder::import_authentication_credentials(…​)</code></p> +</li> +<li> +<p><code>KuduClient::ExportAuthenticationCredentials(…​)</code></p> +</li> +<li> +<p><code>KuduClient::NewIsNotNullPredicate(…​)</code></p> </li> <li> -<p><code>KuduSession::SetErrorBufferSpace(…​)</code></p> +<p><code>KuduClient::NewIsNullPredicate(…​)</code></p> </li> </ul> </div> </li> <li> -<p>The Kudu 1.2 Python client is API-compatible with Kudu 1.1. Applications -written against Kudu 1.1 will continue to run against the Kudu 1.2 client +<p>The Kudu 1.3 Python client is API-compatible with Kudu 1.2. Applications +written against Kudu 1.2 will continue to run against the Kudu 1.3 client and vice-versa.</p> </li> </ul> @@ -319,7 +298,7 @@ and vice-versa.</p> </div> </div> <div class="sect1"> -<h2 id="rn_1.2.0_known_issues"><a class="link" href="#rn_1.2.0_known_issues">Known Issues and Limitations</a></h2> +<h2 id="rn_1.3.0_known_issues"><a class="link" href="#rn_1.3.0_known_issues">Known Issues and Limitations</a></h2> <div class="sectionbody"> <div class="paragraph"> <p>Please refer to the <a href="known_issues.html">Known Issues and Limitations</a> section of the @@ -387,17 +366,16 @@ documentation.</p> <li> <span class="active-toc">Kudu Release Notes</span> <ul class="sectlevel1"> -<li><a href="#rn_1.2.0_new_features">New features</a></li> +<li><a href="#rn_1.3.0_new_features">New features</a></li> <li><a href="#_optimizations_and_improvements">Optimizations and improvements</a></li> -<li><a href="#rn_1.2.0_fixed_issues">Fixed Issues</a></li> -<li><a href="#rn_1.2.0_wire_compatibility">Wire Protocol compatibility</a></li> -<li><a href="#rn_1.2.0_incompatible_changes">Incompatible Changes in Kudu 1.2.0</a> +<li><a href="#rn_1.3.0_fixed_issues">Fixed Issues</a></li> +<li><a href="#rn_1.3.0_wire_compatibility">Wire Protocol compatibility</a></li> +<li><a href="#rn_1.3.0_incompatible_changes">Incompatible Changes in Kudu 1.3.0</a> <ul class="sectlevel2"> -<li><a href="#_new_restrictions_on_data_schemas_and_identifiers">New Restrictions on Data, Schemas, and Identifiers</a></li> -<li><a href="#rn_1.2.0_client_compatibility">Client Library Compatibility</a></li> +<li><a href="#rn_1.3.0_client_compatibility">Client Library Compatibility</a></li> </ul> </li> -<li><a href="#rn_1.2.0_known_issues">Known Issues and Limitations</a></li> +<li><a href="#rn_1.3.0_known_issues">Known Issues and Limitations</a></li> <li><a href="#resources_and_next_steps">Resources</a></li> <li><a href="#_installation_options">Installation Options</a></li> <li><a href="#_next_steps">Next Steps</a></li> http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/schema_design.html ---------------------------------------------------------------------- diff --git a/docs/schema_design.html b/docs/schema_design.html index ad6398e..0f6e19a 100644 --- a/docs/schema_design.html +++ b/docs/schema_design.html @@ -2,7 +2,7 @@ title: Apache Kudu Schema Design layout: default active_nav: docs -last_updated: 'Last updated 2017-01-26 16:14:09 PST' +last_updated: 'Last updated 2017-03-01 12:43:33 PST' --- <!-- @@ -130,40 +130,47 @@ be specified on a per-column basis.</p> <h3 id="encoding"><a class="link" href="#encoding">Column Encoding</a></h3> <div class="paragraph"> <p>Each column in a Kudu table can be created with an encoding, based on the type -of the column. Columns use plain encoding by default.</p> +of the column.</p> </div> <table class="tableblock frame-all grid-all spread"> <caption class="title">Table 1. Encoding Types</caption> <colgroup> -<col style="width: 50%;"> -<col style="width: 50%;"> +<col style="width: 33%;"> +<col style="width: 33%;"> +<col style="width: 33%;"> </colgroup> <thead> <tr> <th class="tableblock halign-left valign-top">Column Type</th> <th class="tableblock halign-left valign-top">Encoding</th> +<th class="tableblock halign-left valign-top">Default</th> </tr> </thead> <tbody> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">int8, int16, int32</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">plain, bitshuffle, run length</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">bitshuffle</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">int64, unixtime_micros</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">plain, bitshuffle</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">plain, bitshuffle, run length</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">bitshuffle</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">float, double</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">plain, bitshuffle</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">bitshuffle</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">bool</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">plain, run length</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">run length</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">string, binary</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">plain, prefix, dictionary</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">dictionary</p></td> </tr> </tbody> </table> @@ -454,7 +461,7 @@ independently.</p> <div class="sect2"> <h3 id="partitioning-examples"><a class="link" href="#partitioning-examples">Partitioning Examples</a></h3> <div class="paragraph"> -<p>To illustrate the factors and tradeoffs associated with designing a partitioning +<p>To illustrate the factors and trade-offs associated with designing a partitioning strategy for a table, we will walk through some different partitioning scenarios. Consider the following table schema for storing machine metrics data (using SQL syntax and date-formatted timestamps for clarity):</p> @@ -692,7 +699,7 @@ performance.</p> </dd> <dt class="hdlist1">Size of Cells</dt> <dd> -<p>No individual cell may be larger than 64KB. The cells making up a +<p>No individual cell may be larger than 64KB. The cells making up a composite key are limited to a total of 16KB after the internal composite-key encoding done by Kudu. Inserting rows not conforming to these limitations will result in errors being returned to the client.</p> http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/style_guide.html ---------------------------------------------------------------------- diff --git a/docs/style_guide.html b/docs/style_guide.html index 9347925..646554b 100644 --- a/docs/style_guide.html +++ b/docs/style_guide.html @@ -2,7 +2,7 @@ title: Apache Kudu Documentation Style Guide layout: default active_nav: docs -last_updated: 'Last updated 2016-08-30 11:05:39 PDT' +last_updated: 'Last updated 2017-02-03 10:13:21 PST' --- <!-- http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/transaction_semantics.html ---------------------------------------------------------------------- diff --git a/docs/transaction_semantics.html b/docs/transaction_semantics.html index 5849f19..77ebac5 100644 --- a/docs/transaction_semantics.html +++ b/docs/transaction_semantics.html @@ -2,7 +2,7 @@ title: Transaction Semantics in Apache Kudu layout: default active_nav: docs -last_updated: 'Last updated 2017-01-26 16:14:09 PST' +last_updated: 'Last updated 2017-02-03 10:13:21 PST' --- <!-- http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/docs/troubleshooting.html ---------------------------------------------------------------------- diff --git a/docs/troubleshooting.html b/docs/troubleshooting.html index 7267b51..325458a 100644 --- a/docs/troubleshooting.html +++ b/docs/troubleshooting.html @@ -2,7 +2,7 @@ title: Apache Kudu Troubleshooting layout: default active_nav: docs -last_updated: 'Last updated 2016-08-30 11:05:39 PDT' +last_updated: 'Last updated 2017-02-03 10:13:21 PST' --- <!--
