Grant Henke created KUDU-3211:
---------------------------------
Summary: Add a cluster supported features request
Key: KUDU-3211
URL: https://issues.apache.org/jira/browse/KUDU-3211
Project: Kudu
Issue Type: Improvement
Components: master, supportability
Affects Versions: 1.13.0
Reporter: Grant Henke
Assignee: Grant Henke
Recently we have come across a few scenarios where it would be useful to make
decisions in client integrations (Backup/Restore, Spark, NiFi, Impala) based on
the supported features of the target Kudu cluster. This can especially helpful
when we want to use new features by default if available but using the new
feature requires client/integration logic changes.
Some recent examples:
- Push bloomfilter predicates only if supported
- Use insert ignore operations (vs session based ignore) only if supported
It is technically possible to be optimistic about the support of a feature and
try to handle errors in a clever way using the required feature capabilities of
the RPCs. However, that can be difficult to express and near impossible if you
want to make a decision for multiple requests or based on what all tablet
servers support instead of based on a single request to a single tablet server.
Additionally now that we support rolling restart, we can't assume that because
a single master or tablet server supports a feature that all servers in the
cluster support the feature.
Some thoughts on the feature/implementation:
- This should be a master request in order to prevent needing to talk to all
the tablet servers.
- We could leverage server registration requests or heartbeats to aggregate the
current state on the leader master.
- We could represent these features as "cluster" level features and indicate
that some (union) or all (intersect) of the servers support a given feature.
- If this request/response is not available in a cluster the response would
indicate that feature support is unknown and the user can decide how to proceed.
- If we want to support disabling features via runtime flags we will need to
ensure we update the master, maybe via heartbeat, with changed support for a
running server.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
