brian remedios created FLUME-2506:
-------------------------------------
Summary: LoadBalancingRpcClient should allow for preferred hosts
in selector
Key: FLUME-2506
URL: https://issues.apache.org/jira/browse/FLUME-2506
Project: Flume
Issue Type: Improvement
Components: Client SDK
Reporter: brian remedios
The current HostSelector interface relies on an iterator to return the next
host without regard to the incoming event data. For clients that use internal
caches to speed processing or were otherwise optimized for specific events, it
would be nice if host selection could depend on a quick examination by the
selector beforehand. Allowing support for host preferences (stickiness) would
keep individual caches much tighter but would necessitate replacing the
iterator with something that is provided with the actual event:
{code}
public void append(Event event) throws EventDeliveryException {
boolean eventSent = false;
HostProvider provider = selector.getHostProvider();
HostInfo info = provider.hostFor(event); // host-specific per event
int attempts = 0;
while (info != null && attempts < MAX) {
RpcClient client;
attempts++;
try {
client = getClient(host);
client.append(event);
eventSent = true;
break;
} catch (Exception ex) {
selector.informFailure(host);
LOGGER.warn("Failed to send event to host " + host, ex);
}
info = provider.hostFor(event); // try another (if available)
}
{code}
The new HostProvider interface could wrap the current round-robin and random
allocation strategies by default to avoid breaking existing code while others
could take advantage of event specificity to chose the host.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
