[
https://issues.apache.org/jira/browse/HADOOP-11620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337467#comment-14337467
]
Arun Suresh commented on HADOOP-11620:
--------------------------------------
[~andrew.wang], sorry, I guess I missed to clarify a couple of things in my
previous comment :
bq. Thinking about it a bit too, are we worried about convoy effects? With a
bunch of clients doing RR, if one KMS goes down, all the clients are going to
pile up until they timeout on that one KMS. Then the next KMS in the list is
going to get hammered when all the clients try the next one.
So, there are a couple of things here :
1) if a bunch of clients are using the same LBKMSClientProvider object : The
{{nextIdx()}} method returns a {{currentIdx+1}} for exactly this reason. This
will ensure each concurrent client of the LBKMSClientProvider will get a
different provider..
2) Bunch of distributed clients issuing requests on different
LBKMSClientProvider objects : This would be rare unless all of them had created
the ClientProvider at the same (systime % numProvider)... but since the number
of KMSs hosts in the group are small (likely 2-3), this might be more common..
but then randomizing the list also might not help.. will it ?
bq. I see it's also still returning the currentIdx+1 rather than currentIdx,..
Hope the above explanation clarifies this.
> Add Support for Load Balancing across a group of KMS servers for HA
> -------------------------------------------------------------------
>
> Key: HADOOP-11620
> URL: https://issues.apache.org/jira/browse/HADOOP-11620
> Project: Hadoop Common
> Issue Type: Improvement
> Components: kms
> Affects Versions: 2.6.0
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Attachments: HADOOP-11620.1.patch, HADOOP-11620.2.patch,
> HADOOP-11620.3.patch, HADOOP-11620.4.patch, HADOOP-11620.5.patch
>
>
> This patch needs to add support for :
> * specification of multiple hostnames in the kms key provider uri
> * KMS client to load balance requests across the hosts specified in the kms
> keyprovider uri.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
