[04/50] [abbrv] hadoop git commit: HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang.

2016-08-18 Thread cnauroth 
HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. 
Contributed by Wei-Chiu Chuang.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/db719ef1
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/db719ef1
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/db719ef1

Branch: refs/heads/HADOOP-13345
Commit: db719ef125b11b01eab3353e2dc4b48992bf88d5
Parents: d892ae9
Author: Wei-Chiu Chuang 
Authored: Thu Aug 11 12:27:09 2016 -0700
Committer: Wei-Chiu Chuang 
Committed: Thu Aug 11 12:27:09 2016 -0700

--
 .../hadoop-kms/src/site/markdown/index.md.vm| 66 +---
 1 file changed, 59 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hadoop/blob/db719ef1/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
--
diff --git a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm 
b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
index d50b0f4..889dbaf1 100644
--- a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
+++ b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
@@ -19,6 +19,8 @@
 Hadoop Key Management Server (KMS) - Documentation Sets
 ===
 
+
+
 Hadoop KMS is a cryptographic key management server based on Hadoop's 
**KeyProvider** API.
 
 It provides a client and a server components which communicate over HTTP using 
a REST API.
@@ -34,6 +36,18 @@ KMS Client Configuration
 
 The KMS client `KeyProvider` uses the **kms** scheme, and the embedded URL 
must be the URL of the KMS. For example, for a KMS running on 
`http://localhost:9600/kms`, the KeyProvider URI is 
`kms://http@localhost:9600/kms`. And, for a KMS running on 
`https://localhost:9600/kms`, the KeyProvider URI is 
`kms://https@localhost:9600/kms`
 
+The following is an example to configure HDFS NameNode as a KMS client in
+`hdfs-site.xml`:
+
+
+  dfs.encryption.key.provider.uri
+  kms://http@localhost:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
 KMS
 ---
 
@@ -623,13 +637,51 @@ Additionally, KMS delegation token secret manager can be 
configured with the fol
   
 ```
 
-$H3 Using Multiple Instances of KMS Behind a Load-Balancer or VIP
-
-KMS supports multiple KMS instances behind a load-balancer or VIP for 
scalability and for HA purposes.
-
-When using multiple KMS instances behind a load-balancer or VIP, requests from 
the same user may be handled by different KMS instances.
-
-KMS instances behind a load-balancer or VIP must be specially configured to 
work properly as a single logical service.
+$H3 High Availability
+
+Multiple KMS instances may be used to provide high availability and 
scalability.
+Currently there are two approaches to supporting multiple KMS instances:
+running KMS instances behind a load-balancer/VIP,
+or using LoadBalancingKMSClientProvider.
+
+In both approaches, KMS instances must be specially configured to work properly
+as a single logical service, because requests from the same client may be
+handled by different KMS instances. In particular,
+Kerberos Principals Configuration, HTTP Authentication Signature and Delegation
+Tokens require special attention.
+
+$H4 Behind a Load-Balancer or VIP
+
+Because KMS clients and servers communicate via a REST API over HTTP,
+Load-balancer or VIP may be used to distribute incoming traffic to achieve
+scalability and HA. In this mode, clients are unaware of multiple KMS instances
+at the server-side.
+
+$H4 Using LoadBalancingKMSClientProvider
+
+An alternative to running multiple KMS instances behind a load-balancer or VIP,
+is to use LoadBalancingKMSClientProvider. Using this approach, a KMS client
+(for example, a HDFS NameNode) is aware of multiple KMS instances, and it sends
+requests to them in a round-robin fashion. LoadBalancingKMSClientProvider is
+implicitly used when more than one URI is specified in
+`dfs.encryption.key.provider.uri`.
+
+The following example in `hdfs-site.xml` configures two KMS
+instances, `kms01.example.com` and `kms02.example.com`.
+The hostnames are separated by semi-colons, and all KMS instances must run
+on the same port.
+
+
+  dfs.encryption.key.provider.uri
+  kms://ht...@kms01.example.com;kms02.example.com:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
+If a request to a KMS instance fails, clients retry with the next instance. The
+request is returned as failure only if all instances

[02/50] [abbrv] hadoop git commit: HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang.

2016-08-17 Thread subru 
HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. 
Contributed by Wei-Chiu Chuang.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/db719ef1
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/db719ef1
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/db719ef1

Branch: refs/heads/YARN-2915
Commit: db719ef125b11b01eab3353e2dc4b48992bf88d5
Parents: d892ae9
Author: Wei-Chiu Chuang 
Authored: Thu Aug 11 12:27:09 2016 -0700
Committer: Wei-Chiu Chuang 
Committed: Thu Aug 11 12:27:09 2016 -0700

--
 .../hadoop-kms/src/site/markdown/index.md.vm| 66 +---
 1 file changed, 59 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hadoop/blob/db719ef1/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
--
diff --git a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm 
b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
index d50b0f4..889dbaf1 100644
--- a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
+++ b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
@@ -19,6 +19,8 @@
 Hadoop Key Management Server (KMS) - Documentation Sets
 ===
 
+
+
 Hadoop KMS is a cryptographic key management server based on Hadoop's 
**KeyProvider** API.
 
 It provides a client and a server components which communicate over HTTP using 
a REST API.
@@ -34,6 +36,18 @@ KMS Client Configuration
 
 The KMS client `KeyProvider` uses the **kms** scheme, and the embedded URL 
must be the URL of the KMS. For example, for a KMS running on 
`http://localhost:9600/kms`, the KeyProvider URI is 
`kms://http@localhost:9600/kms`. And, for a KMS running on 
`https://localhost:9600/kms`, the KeyProvider URI is 
`kms://https@localhost:9600/kms`
 
+The following is an example to configure HDFS NameNode as a KMS client in
+`hdfs-site.xml`:
+
+
+  dfs.encryption.key.provider.uri
+  kms://http@localhost:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
 KMS
 ---
 
@@ -623,13 +637,51 @@ Additionally, KMS delegation token secret manager can be 
configured with the fol
   
 ```
 
-$H3 Using Multiple Instances of KMS Behind a Load-Balancer or VIP
-
-KMS supports multiple KMS instances behind a load-balancer or VIP for 
scalability and for HA purposes.
-
-When using multiple KMS instances behind a load-balancer or VIP, requests from 
the same user may be handled by different KMS instances.
-
-KMS instances behind a load-balancer or VIP must be specially configured to 
work properly as a single logical service.
+$H3 High Availability
+
+Multiple KMS instances may be used to provide high availability and 
scalability.
+Currently there are two approaches to supporting multiple KMS instances:
+running KMS instances behind a load-balancer/VIP,
+or using LoadBalancingKMSClientProvider.
+
+In both approaches, KMS instances must be specially configured to work properly
+as a single logical service, because requests from the same client may be
+handled by different KMS instances. In particular,
+Kerberos Principals Configuration, HTTP Authentication Signature and Delegation
+Tokens require special attention.
+
+$H4 Behind a Load-Balancer or VIP
+
+Because KMS clients and servers communicate via a REST API over HTTP,
+Load-balancer or VIP may be used to distribute incoming traffic to achieve
+scalability and HA. In this mode, clients are unaware of multiple KMS instances
+at the server-side.
+
+$H4 Using LoadBalancingKMSClientProvider
+
+An alternative to running multiple KMS instances behind a load-balancer or VIP,
+is to use LoadBalancingKMSClientProvider. Using this approach, a KMS client
+(for example, a HDFS NameNode) is aware of multiple KMS instances, and it sends
+requests to them in a round-robin fashion. LoadBalancingKMSClientProvider is
+implicitly used when more than one URI is specified in
+`dfs.encryption.key.provider.uri`.
+
+The following example in `hdfs-site.xml` configures two KMS
+instances, `kms01.example.com` and `kms02.example.com`.
+The hostnames are separated by semi-colons, and all KMS instances must run
+on the same port.
+
+
+  dfs.encryption.key.provider.uri
+  kms://ht...@kms01.example.com;kms02.example.com:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
+If a request to a KMS instance fails, clients retry with the next instance. The
+request is returned as failure only if all instances

hadoop git commit: HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang.

2016-08-11 Thread weichiu 
Repository: hadoop
Updated Branches:
  refs/heads/branch-2.8 f7cc0af61 -> b0bc7681c


HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit db719ef125b11b01eab3353e2dc4b48992bf88d5)
(cherry picked from commit 6d53e096b2d96b9aace46fa9f34cd93e7f0f22a6)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/b0bc7681
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/b0bc7681
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/b0bc7681

Branch: refs/heads/branch-2.8
Commit: b0bc7681c32c33756a6c4c77aa9257990183dee6
Parents: f7cc0af
Author: Wei-Chiu Chuang 
Authored: Thu Aug 11 12:27:09 2016 -0700
Committer: Wei-Chiu Chuang 
Committed: Thu Aug 11 12:33:15 2016 -0700

--
 .../hadoop-kms/src/site/markdown/index.md.vm| 66 +---
 1 file changed, 59 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hadoop/blob/b0bc7681/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
--
diff --git a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm 
b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
index d2d1d99..729b2b9 100644
--- a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
+++ b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
@@ -19,6 +19,8 @@
 Hadoop Key Management Server (KMS) - Documentation Sets
 ===
 
+
+
 Hadoop KMS is a cryptographic key management server based on Hadoop's 
**KeyProvider** API.
 
 It provides a client and a server components which communicate over HTTP using 
a REST API.
@@ -34,6 +36,18 @@ KMS Client Configuration
 
 The KMS client `KeyProvider` uses the **kms** scheme, and the embedded URL 
must be the URL of the KMS. For example, for a KMS running on 
`http://localhost:16000/kms`, the KeyProvider URI is 
`kms://http@localhost:16000/kms`. And, for a KMS running on 
`https://localhost:16000/kms`, the KeyProvider URI is 
`kms://https@localhost:16000/kms`
 
+The following is an example to configure HDFS NameNode as a KMS client in
+`hdfs-site.xml`:
+
+
+  dfs.encryption.key.provider.uri
+  kms://http@localhost:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
 KMS
 ---
 
@@ -623,13 +637,51 @@ Additionally, KMS delegation token secret manager can be 
configured with the fol
   
 ```
 
-$H3 Using Multiple Instances of KMS Behind a Load-Balancer or VIP
-
-KMS supports multiple KMS instances behind a load-balancer or VIP for 
scalability and for HA purposes.
-
-When using multiple KMS instances behind a load-balancer or VIP, requests from 
the same user may be handled by different KMS instances.
-
-KMS instances behind a load-balancer or VIP must be specially configured to 
work properly as a single logical service.
+$H3 High Availability
+
+Multiple KMS instances may be used to provide high availability and 
scalability.
+Currently there are two approaches to supporting multiple KMS instances:
+running KMS instances behind a load-balancer/VIP,
+or using LoadBalancingKMSClientProvider.
+
+In both approaches, KMS instances must be specially configured to work properly
+as a single logical service, because requests from the same client may be
+handled by different KMS instances. In particular,
+Kerberos Principals Configuration, HTTP Authentication Signature and Delegation
+Tokens require special attention.
+
+$H4 Behind a Load-Balancer or VIP
+
+Because KMS clients and servers communicate via a REST API over HTTP,
+Load-balancer or VIP may be used to distribute incoming traffic to achieve
+scalability and HA. In this mode, clients are unaware of multiple KMS instances
+at the server-side.
+
+$H4 Using LoadBalancingKMSClientProvider
+
+An alternative to running multiple KMS instances behind a load-balancer or VIP,
+is to use LoadBalancingKMSClientProvider. Using this approach, a KMS client
+(for example, a HDFS NameNode) is aware of multiple KMS instances, and it sends
+requests to them in a round-robin fashion. LoadBalancingKMSClientProvider is
+implicitly used when more than one URI is specified in
+`dfs.encryption.key.provider.uri`.
+
+The following example in `hdfs-site.xml` configures two KMS
+instances, `kms01.example.com` and `kms02.example.com`.
+The hostnames are separated by semi-colons, and all KMS instances must run
+on the same port.
+
+
+  dfs.encryption.key.provider.uri
+  kms://ht...@kms01.example.com;kms02.example.com:9600/kms
+  
+The KeyProvider to use when interacting with

hadoop git commit: HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang.

2016-08-11 Thread weichiu 
Repository: hadoop
Updated Branches:
  refs/heads/branch-2 d4501ad0d -> 6d53e096b


HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit db719ef125b11b01eab3353e2dc4b48992bf88d5)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/6d53e096
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/6d53e096
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/6d53e096

Branch: refs/heads/branch-2
Commit: 6d53e096b2d96b9aace46fa9f34cd93e7f0f22a6
Parents: d4501ad
Author: Wei-Chiu Chuang 
Authored: Thu Aug 11 12:27:09 2016 -0700
Committer: Wei-Chiu Chuang 
Committed: Thu Aug 11 12:27:52 2016 -0700

--
 .../hadoop-kms/src/site/markdown/index.md.vm| 66 +---
 1 file changed, 59 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hadoop/blob/6d53e096/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
--
diff --git a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm 
b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
index d2d1d99..729b2b9 100644
--- a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
+++ b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
@@ -19,6 +19,8 @@
 Hadoop Key Management Server (KMS) - Documentation Sets
 ===
 
+
+
 Hadoop KMS is a cryptographic key management server based on Hadoop's 
**KeyProvider** API.
 
 It provides a client and a server components which communicate over HTTP using 
a REST API.
@@ -34,6 +36,18 @@ KMS Client Configuration
 
 The KMS client `KeyProvider` uses the **kms** scheme, and the embedded URL 
must be the URL of the KMS. For example, for a KMS running on 
`http://localhost:16000/kms`, the KeyProvider URI is 
`kms://http@localhost:16000/kms`. And, for a KMS running on 
`https://localhost:16000/kms`, the KeyProvider URI is 
`kms://https@localhost:16000/kms`
 
+The following is an example to configure HDFS NameNode as a KMS client in
+`hdfs-site.xml`:
+
+
+  dfs.encryption.key.provider.uri
+  kms://http@localhost:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
 KMS
 ---
 
@@ -623,13 +637,51 @@ Additionally, KMS delegation token secret manager can be 
configured with the fol
   
 ```
 
-$H3 Using Multiple Instances of KMS Behind a Load-Balancer or VIP
-
-KMS supports multiple KMS instances behind a load-balancer or VIP for 
scalability and for HA purposes.
-
-When using multiple KMS instances behind a load-balancer or VIP, requests from 
the same user may be handled by different KMS instances.
-
-KMS instances behind a load-balancer or VIP must be specially configured to 
work properly as a single logical service.
+$H3 High Availability
+
+Multiple KMS instances may be used to provide high availability and 
scalability.
+Currently there are two approaches to supporting multiple KMS instances:
+running KMS instances behind a load-balancer/VIP,
+or using LoadBalancingKMSClientProvider.
+
+In both approaches, KMS instances must be specially configured to work properly
+as a single logical service, because requests from the same client may be
+handled by different KMS instances. In particular,
+Kerberos Principals Configuration, HTTP Authentication Signature and Delegation
+Tokens require special attention.
+
+$H4 Behind a Load-Balancer or VIP
+
+Because KMS clients and servers communicate via a REST API over HTTP,
+Load-balancer or VIP may be used to distribute incoming traffic to achieve
+scalability and HA. In this mode, clients are unaware of multiple KMS instances
+at the server-side.
+
+$H4 Using LoadBalancingKMSClientProvider
+
+An alternative to running multiple KMS instances behind a load-balancer or VIP,
+is to use LoadBalancingKMSClientProvider. Using this approach, a KMS client
+(for example, a HDFS NameNode) is aware of multiple KMS instances, and it sends
+requests to them in a round-robin fashion. LoadBalancingKMSClientProvider is
+implicitly used when more than one URI is specified in
+`dfs.encryption.key.provider.uri`.
+
+The following example in `hdfs-site.xml` configures two KMS
+instances, `kms01.example.com` and `kms02.example.com`.
+The hostnames are separated by semi-colons, and all KMS instances must run
+on the same port.
+
+
+  dfs.encryption.key.provider.uri
+  kms://ht...@kms01.example.com;kms02.example.com:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.

hadoop git commit: HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang.

2016-08-11 Thread weichiu 
Repository: hadoop
Updated Branches:
  refs/heads/trunk d892ae957 -> db719ef12


HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. 
Contributed by Wei-Chiu Chuang.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/db719ef1
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/db719ef1
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/db719ef1

Branch: refs/heads/trunk
Commit: db719ef125b11b01eab3353e2dc4b48992bf88d5
Parents: d892ae9
Author: Wei-Chiu Chuang 
Authored: Thu Aug 11 12:27:09 2016 -0700
Committer: Wei-Chiu Chuang 
Committed: Thu Aug 11 12:27:09 2016 -0700

--
 .../hadoop-kms/src/site/markdown/index.md.vm| 66 +---
 1 file changed, 59 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hadoop/blob/db719ef1/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
--
diff --git a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm 
b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
index d50b0f4..889dbaf1 100644
--- a/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
+++ b/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
@@ -19,6 +19,8 @@
 Hadoop Key Management Server (KMS) - Documentation Sets
 ===
 
+
+
 Hadoop KMS is a cryptographic key management server based on Hadoop's 
**KeyProvider** API.
 
 It provides a client and a server components which communicate over HTTP using 
a REST API.
@@ -34,6 +36,18 @@ KMS Client Configuration
 
 The KMS client `KeyProvider` uses the **kms** scheme, and the embedded URL 
must be the URL of the KMS. For example, for a KMS running on 
`http://localhost:9600/kms`, the KeyProvider URI is 
`kms://http@localhost:9600/kms`. And, for a KMS running on 
`https://localhost:9600/kms`, the KeyProvider URI is 
`kms://https@localhost:9600/kms`
 
+The following is an example to configure HDFS NameNode as a KMS client in
+`hdfs-site.xml`:
+
+
+  dfs.encryption.key.provider.uri
+  kms://http@localhost:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
 KMS
 ---
 
@@ -623,13 +637,51 @@ Additionally, KMS delegation token secret manager can be 
configured with the fol
   
 ```
 
-$H3 Using Multiple Instances of KMS Behind a Load-Balancer or VIP
-
-KMS supports multiple KMS instances behind a load-balancer or VIP for 
scalability and for HA purposes.
-
-When using multiple KMS instances behind a load-balancer or VIP, requests from 
the same user may be handled by different KMS instances.
-
-KMS instances behind a load-balancer or VIP must be specially configured to 
work properly as a single logical service.
+$H3 High Availability
+
+Multiple KMS instances may be used to provide high availability and 
scalability.
+Currently there are two approaches to supporting multiple KMS instances:
+running KMS instances behind a load-balancer/VIP,
+or using LoadBalancingKMSClientProvider.
+
+In both approaches, KMS instances must be specially configured to work properly
+as a single logical service, because requests from the same client may be
+handled by different KMS instances. In particular,
+Kerberos Principals Configuration, HTTP Authentication Signature and Delegation
+Tokens require special attention.
+
+$H4 Behind a Load-Balancer or VIP
+
+Because KMS clients and servers communicate via a REST API over HTTP,
+Load-balancer or VIP may be used to distribute incoming traffic to achieve
+scalability and HA. In this mode, clients are unaware of multiple KMS instances
+at the server-side.
+
+$H4 Using LoadBalancingKMSClientProvider
+
+An alternative to running multiple KMS instances behind a load-balancer or VIP,
+is to use LoadBalancingKMSClientProvider. Using this approach, a KMS client
+(for example, a HDFS NameNode) is aware of multiple KMS instances, and it sends
+requests to them in a round-robin fashion. LoadBalancingKMSClientProvider is
+implicitly used when more than one URI is specified in
+`dfs.encryption.key.provider.uri`.
+
+The following example in `hdfs-site.xml` configures two KMS
+instances, `kms01.example.com` and `kms02.example.com`.
+The hostnames are separated by semi-colons, and all KMS instances must run
+on the same port.
+
+
+  dfs.encryption.key.provider.uri
+  kms://ht...@kms01.example.com;kms02.example.com:9600/kms
+  
+The KeyProvider to use when interacting with encryption keys used
+when reading and writing to an encryption zone.
+  
+
+
+If a request to a KMS instance fails, clients retry with the