This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch branch-1.14.x
in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/branch-1.14.x by this push:
new e0a81ab Add release notes for 1.14.0
e0a81ab is described below
commit e0a81abc2f503284b6e1a86b92678e5d82c9bbc1
Author: Grant Henke <[email protected]>
AuthorDate: Mon Jan 25 13:33:33 2021 -0600
Add release notes for 1.14.0
Change-Id: Ib8da030ce0f8797f8c366c56298fcda1020b00b6
Reviewed-on: http://gerrit.cloudera.org:8080/16980
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <[email protected]>
Reviewed-by: Greg Solovyev <[email protected]>
---
docs/release_notes.adoc | 218 +++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 187 insertions(+), 31 deletions(-)
diff --git a/docs/release_notes.adoc b/docs/release_notes.adoc
index f23e070..212cae6 100644
--- a/docs/release_notes.adoc
+++ b/docs/release_notes.adoc
@@ -28,82 +28,238 @@
:sectlinks:
:experimental:
-[[rn_1.13.0_upgrade_notes]]
+[[rn_1.14.0_upgrade_notes]]
== Upgrade Notes
-
-[[rn_1.13.0_obsoletions]]
+[[rn_1.14.0_obsoletions]]
== Obsoletions
+* Support for CentOS 6/RHEL 6, Ubuntu 14, Ubuntu 16, and Debian 8 platforms
has been dropped
+ given they are at or near end-of-life. We will no longer validate these
platforms as a
+ part of the release process, though patches will still be accepted going
forward.
+
+* Developer support for OS X 10.10 Yosemite, OS X 10.11 El Capitan, and OS X
10.12 Sierra
+ has been dropped. We will no longer validate these versions as a part of the
release
+ process, though patches will still be accepted going forward.
-[[rn_1.13.0_deprecations]]
+[[rn_1.14.0_deprecations]]
== Deprecations
-Support for Python 2.x and Python 3.4 and earlier is deprecated and may be
removed in the next minor release.
+* Support for Python 2.x and Python 3.4 and earlier is deprecated and may be
+ removed in the next minor release.
-[[rn_1.13.0_new_features]]
-== New features
+* The `kudu-mapreduce` integration has been deprecated and may be removed in
the
+ next minor release. Similar functionality and capabilities now exist via the
+ Apache Spark, Apache Hive, Apache Impala, and Apache NiFi integrations.
+[[rn_1.14.0_new_features]]
+== New features
-[[rn_1.13.0_improvements]]
+* Full support for `INSERT_IGNORE`, `UPDATE_IGNORE`, and `DELETE_IGNORE`
operations
+ was added. The `INSERT_IGNORE` operation will insert a row if one matching
the key
+ does not exist and ignore the operation if one already exists. The
`UPDATE_IGNORE`
+ operation will update the row if one matching the key exists and ignore the
operation
+ if one does not exist. The `DELETE_IGNORE` operation will delete the row if
one matching
+ the key exists and ignore the operation if one does not exist. These
operations are
+ particularly useful in situations where retries or duplicate operations
could occur and
+ you do not want to handle the errors that could result manually or you do
not want to cause
+ unnecessary writes and compaction work as a result of using the `UPSERT`
operation.
+ The Java client can check if the cluster it is communicating with supports
these operations
+ by calling the `supportsIgnoreOperations()` method on the KuduClient. See
+ link:https://issues.apache.org/jira/browse/KUDU-1563[KUDU-1563] for more
details.
+
+* Spark 3 compatible JARs compiled for Scala 2.12 are now published for the
Kudu Spark integration.
+ See link:https://issues.apache.org/jira/browse/KUDU-3202[KUDU-3202] for more
details.
+
+* Every Kudu cluster now has an automatically generated cluster Id that can be
used to uniquely
+ identify a cluster. The cluster Id is shown in the masters web-UI, the `kudu
master list` tool,
+ and in master server logs. See
link:https://issues.apache.org/jira/browse/KUDU-2574[KUDU-2574]
+ for more details.
+
+* It is now possible to enforce that OpenSSL is initialized in FIPS approved
mode in the servers
+ and the C++ client by setting the KUDU_REQUIRE_FIPS_MODE environment
variable to “1”, “yes” or
+ “true”. See link:https://issues.apache.org/jira/browse/KUDU-3210[KUDU-3210]
for more details.
+
+[[rn_1.14.0_improvements]]
== Optimizations and improvements
-
-[[rn_1.13.0_fixed_issues]]
+* Downloading the WAL data and data blocks when copying tablets to another
tablet server is now
+ parallelized, resulting in much faster tablet copy operations. These
operations occur when
+ recovering from a down tablet server or when running the cluster rebalancer.
See
+ link:https://issues.apache.org/jira/browse/KUDU-1728[KUDU-1728] and
+ link:https://issues.apache.org/jira/browse/KUDU-3214[KUDU-3214] for more
details.
+
+* The HMS integration now supports multiple Kudu clusters associated with a
single HMS
+ including Kudu clusters that do not have HMS synchronization enabled. This
is possible,
+ because the Kudu master will now leverage the cluster Id to ignore
notifications from
+ tables in a different cluster. Additionally, the HMS plugin will check if
the Kudu cluster
+ associated with a table has HMS synchronization enabled.
+ See link:https://issues.apache.org/jira/browse/KUDU-3192[KUDU-3192] and
+ link:https://issues.apache.org/jira/browse/KUDU-3187[KUDU-3187] for more
details.
+
+* The HMS integration now supports gzipped HMS notifications. This is
important in order to
+ support Hive 4 where the default encoder was changed to be the
GzipJSONMessageEncoder. See
+ link:https://issues.apache.org/jira/browse/KUDU-3201[KUDU-3201] for more
details.
+
+* Kudu will now fail tablet replicas that have been corrupted due to KUDU-2233
instead of
+ crashing the tablet server. If a healthy majority still exists, a new
replica will be created
+ and the failed replica will be evicted and deleted. See
+ link:https://issues.apache.org/jira/browse/KUDU-3191[KUDU-3191] and
+ link:https://issues.apache.org/jira/browse/KUDU-2233[KUDU-2233] for more
details.
+
+* DeltaMemStores will now be flushed as long as any DMS in a tablet is older
than the point
+ defined by `--flush_threshold_secs`, rather than flushing once every
`--flush_threshold_secs`
+ period. This can reduce memory pressure under update- or delete-heavy
workloads, and lower tablet
+ server restart times following such workloads. See
+ link:https://issues.apache.org/jira/browse/KUDU-3195[KUDU-3195] for more
details.
+
+* The `kudu perf loadgen` CLI tool now supports `UPSERT` for storing the
generated data into
+ the table. To switch to `UPSERT` for row operations (instead of default
`INSERT`), add the
+ `--use_upsert` command-line flag.
+
+* Users can now specify the level of parallelization when copying a tablet
using the
+ `kudu local_replica copy_from_remote` CLI tool by passing the
+ `--tablet_copy_download_threads_nums_per_session` argument.
+
+* The Kudu Masters now discriminate between overlapped and exact duplicate key
ranges when adding
+ new partitions, returning `Status::AlreadyPresent()` for exact range
duplicates and
+ `Status::InvalidArgument()` for otherwise overlapped ones. In prior
releases, the master
+ returned `Status::InvalidArgument()` both in case of duplicate and otherwise
overlapped ranges.
+
+* The handling of an empty list of master addresses in Kudu C++ client has
improved. In prior
+ releases, `KuduClientBuilder::Build()` would hang in `ConnectToCluster()` if
no master addresses
+ were provided. Now, `KuduClientBuilder::Build()` instantly returns
`Status::InvalidArgument()`
+ in such a case.
+
+* The connection negotiation timeout for Kudu C++ client is now
programmatically configurable.
+ To customize the connection negotiation timeout, use the newly introduced
+ `KuduClientBuilder::connection_negotiation_timeout()` method in the Kudu C++
client API.
+
+* All RPC-related `kudu` CLI tools now have `--negotiation_timeout_ms` command
line flag to
+ control the client-side connection negotiation timeout. The default value
for the new flag is
+ set to 3000 milliseconds for backward compatibility. Keep in mind that the
total RPC timeout
+ includes the connection negotiation time, so in general it makes sense to
bump `--timeout_ms`
+ along with `--negotiation_timeout_ms` by the same delta.
+
+* Kudu now reports on slow SASL calls (i.e. calls taking more than 250
milliseconds to complete)
+ when connecting to a server. This is to help diagnose issues like described
in
+ link:https://issues.apache.org/jira/browse/KUDU-3217[KUDU-3217].
+
+* MaintenanceManager now has a new histogram-based
`maintenance_op_find_best_candidate_duration`
+ metric to capture the stats on how long it takes (in microseconds) to find
the best maintenance
+ operation among available candidates. The newly introduced metric can help
in diagnosing
+ conditions where MaintenanceManager seems lagging behind the rate of write
operations in a busy
+ Kudu cluster with many replicas per tablet server.
+
+* The KuduScanToken Java API has been extended with a
`deserializeIntoScannerBuilder()` method that
+ can be used to further customize generated tokens.
+
+* Logging of the error message produced when applying an op while a Java
KuduSession is closed
+ has been throttled. See
+ link:https://issues.apache.org/jira/browse/KUDU-3012[KUDU-3012] for more
details.
+
+* Added a new `uptime` metric for a Kudu server. The metric's value is
reported as the length of
+ the time interval passed from the start of the server, in microseconds.
Knowing the server's
+ uptime, it's easier to interpret and compare metrics reported by different
Kudu servers.
+
+* Documentation for Kudu’s metrics are now automatically generated for each
release and can be seen
+ link:https://kudu.apache.org/docs/metrics_reference.html[here].
+
+[[rn_1.14.0_fixed_issues]]
== Fixed Issues
+* Fixed lock contention between MaintenanceManager op registration and the
scheduling of new
+ maintenance ops. On particularly dense tablet servers, this contention was
previously shown to
+ significantly slow down startup times. See
+ link:https://issues.apache.org/jira/browse/KUDU-3149[KUDU-3149] for more
details.
+
+* Fixed lock contention between MaintenanceManager’s threads performing
already scheduled
+ operations and the scheduler thread itself. This benefits clusters with
heavy ingest/update
+ workloads that have many replicas per tablet server. See
+ link:https://issues.apache.org/jira/browse/KUDU-1954[KUDU-1954] for more
details.
+
+* Fixed a bug in the merge iterator that could result in a crash. This could
surface as a crash
+ when performing ordered or differential scans, particularly when the
underlying data contained
+ deletes and reinserts. See
+ link:https://issues.apache.org/jira/browse/KUDU-3108[KUDU-3108] for more
details.
+
+* Fixed a heap-use-after-free bug in Kudu C++ client that might manifest
itself when altering a
+ table to update the partitioning schema. See
+ link:https://issues.apache.org/jira/browse/KUDU-3238[KUDU-3238] for more
details.
-[[rn_1.13.0_wire_compatibility]]
+* Fixed a bug where building scan tokens would result in a
NullPointerException if a tablet not
+ found error occurred before generating the token. See
+ link:https://issues.apache.org/jira/browse/KUDU-3205[KUDU-3205] for more
details.
+
+* Fixed a bug where a delete operation would fail if the row being deleted
contained exactly
+ 64 columns and all values were set on the row. See
+ link:https://issues.apache.org/jira/browse/KUDU-3198[KUDU-3198] for more
details.
+
+* Fixed a bug where Slf4j classes were shaded into the Spark integration JARs.
See
+ link:https://issues.apache.org/jira/browse/KUDU-3157[KUDU-3157] for more
details.
+
+* Fixed a bug where the 'kudu hms fix' tool mistakenly reports non-matching
master addresses
+ when the addresses are in-fact canonically the same. See
+ link:https://issues.apache.org/jira/browse/KUDU-2884[KUDU-2884] for more
details.
+
+[[rn_1.14.0_wire_compatibility]]
== Wire Protocol compatibility
-Kudu 1.13.0 is wire-compatible with previous versions of Kudu:
+Kudu 1.14.0 is wire-compatible with previous versions of Kudu:
-* Kudu 1.13 clients may connect to servers running Kudu 1.0 or later. If the
client uses
+* Kudu 1.14 clients may connect to servers running Kudu 1.0 or later. If the
client uses
features that are not available on the target server, an error will be
returned.
-* Rolling upgrade between Kudu 1.12 and Kudu 1.13 servers is believed to be
possible
+* Rolling upgrade between Kudu 1.13 and Kudu 1.14 servers is believed to be
possible
though has not been sufficiently tested. Users are encouraged to shut down
all nodes
in the cluster, upgrade the software, and then restart the daemons on the
new version.
-* Kudu 1.0 clients may connect to servers running Kudu 1.13 with the exception
of the
+* Kudu 1.0 clients may connect to servers running Kudu 1.14 with the exception
of the
below-mentioned restrictions regarding secure clusters.
The authentication features introduced in Kudu 1.3 place the following
limitations
-on wire compatibility between Kudu 1.13 and versions earlier than 1.3:
+on wire compatibility between Kudu 1.14 and versions earlier than 1.3:
-* If a Kudu 1.13 cluster is configured with authentication or encryption set
to "required",
+* If a Kudu 1.14 cluster is configured with authentication or encryption set
to "required",
clients older than Kudu 1.3 will be unable to connect.
-* If a Kudu 1.13 cluster is configured with authentication and encryption set
to "optional"
+* If a Kudu 1.14 cluster is configured with authentication and encryption set
to "optional"
or "disabled", older clients will still be able to connect.
-[[rn_1.13.0_incompatible_changes]]
-== Incompatible Changes in Kudu 1.13.0
+[[rn_1.14.0_incompatible_changes]]
+== Incompatible Changes in Kudu 1.14.0
-[[rn_1.13.0_client_compatibility]]
+[[rn_1.14.0_client_compatibility]]
=== Client Library Compatibility
-* The Kudu 1.13 Java client library is API- and ABI-compatible with Kudu 1.12.
Applications
- written against Kudu 1.12 will compile and run against the Kudu 1.13 client
library and
+* The Kudu 1.14 Java client library is API- and ABI-compatible with Kudu 1.13.
Applications
+ written against Kudu 1.13 will compile and run against the Kudu 1.14 client
library and
vice-versa.
-* The Kudu 1.13 {cpp} client is API- and ABI-forward-compatible with Kudu 1.12.
- Applications written and compiled against the Kudu 1.12 client library will
run without
- modification against the Kudu 1.13 client library. Applications written and
compiled
- against the Kudu 1.13 client library will run without modification against
the Kudu 1.12
+* The Kudu 1.14 {cpp} client is API- and ABI-forward-compatible with Kudu 1.13.
+ Applications written and compiled against the Kudu 1.13 client library will
run without
+ modification against the Kudu 1.14 client library. Applications written and
compiled
+ against the Kudu 1.14 client library will run without modification against
the Kudu 1.13
client library.
-* The Kudu 1.13 Python client is API-compatible with Kudu 1.12. Applications
- written against Kudu 1.12 will continue to run against the Kudu 1.13 client
+* The Kudu 1.14 Python client is API-compatible with Kudu 1.13. Applications
+ written against Kudu 1.13 will continue to run against the Kudu 1.14 client
and vice-versa.
-[[rn_1.13.0_known_issues]]
+[[rn_1.14.0_known_issues]]
== Known Issues and Limitations
Please refer to the link:known_issues.html[Known Issues and Limitations]
section of the
documentation.
-[[rn_1.13.0_contributors]]
+[[rn_1.14.0_contributors]]
== Contributors
+Kudu 1.14.0 includes contributions from 12 people, including 1 first-time
+contributors:
+
+* liguohao
+
+Thank you for your contributions!
+
[[resources_and_next_steps]]
== Resources
