[
https://issues.apache.org/jira/browse/FLUME-962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240903#comment-13240903
]
[email protected] commented on FLUME-962:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4380/
-----------------------------------------------------------
(Updated 2012-03-29 01:39:04.251660)
Review request for Flume.
Changes
-------
Suggestions incorporated.
Summary
-------
Submitting an initial cut of FailoverRpcClient that uses the NettyRpcClient
under the hood. In this version, host selection is not exactly the best, please
make suggestions on how to improve it. As of now, the first version will not
have a backoff mechanism to not retry a host for a fixed time etc(as discussed
in the jira). I will add unit tests soon.
Note that the actual "connect" call to a host is hidden from the FailoverClient
(by the Netty client or any other implementation, which we may choose to use
later). Since this connect call is hidden, failure to create a client(the build
function throwing an exception) is not being considered a failure. Only a
failure to append is considered a failure, and counted towards the maximum
number of tries. In other words, as far as the FailoverClient(for that matter,
any implementation of RpcClient interface) would consider an append failure as
failure, not a failure to a build() call - if we want to make sure that a
connect failure also is counted, we should move the connect call to the append
function and keep track of the connection state internally, and not expect any
code depending on an implementation of RpcClient(including other clients which
depend on pre-existing clients) to know that a build call also creates a
connection - this is exactly like a socket implementation, creating a new
socket does not initialize a connection, it is done explicitly.
This addresses bug FLUME-962.
https://issues.apache.org/jira/browse/FLUME-962
Diffs (updated)
-----
flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 3edc563
flume-ng-sdk/src/main/java/org/apache/flume/api/AbstractRpcClient.java
PRE-CREATION
flume-ng-sdk/src/main/java/org/apache/flume/api/FailoverRpcClient.java
PRE-CREATION
flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java
704be7b
flume-ng-clients/flume-ng-log4jappender/src/main/java/org/apache/flume/clients/log4jappender/Log4jAppender.java
5bc0472
flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
965b2ff
flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClient.java a601213
flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientFactory.java 351b5b1
flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java 93bfee9
flume-ng-sdk/src/test/java/org/apache/flume/api/TestFailoverRpcClient.java
PRE-CREATION
flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
a33e9c8
flume-ng-sdk/src/test/java/org/apache/flume/api/TestRpcClientFactory.java
0c94231
Diff: https://reviews.apache.org/r/4380/diff
Testing
-------
Unit tests added for the new functionality
Thanks,
Hari
> Failover capability for Client SDK
> ----------------------------------
>
> Key: FLUME-962
> URL: https://issues.apache.org/jira/browse/FLUME-962
> Project: Flume
> Issue Type: Sub-task
> Affects Versions: v1.0.0
> Reporter: Kathleen Ting
> Assignee: Hari Shreedharan
> Fix For: v1.2.0
>
> Attachments: FLUME-962-2.patch, FLUME-962-2.patch, FLUME-962-3.patch,
> FLUME-962-3.patch, FLUME-962-4.patch, FLUME-962-5.patch, FLUME-962-6.patch,
> FLUME-962-rebased-1.patch, FLUME-962-rebased-4.patch
>
>
> Need a client SDK for Flume that will allow clients to be able to failover
> from one source to another in case the first agent is not available. This
> will help in keeping client implementations developed outside of the project
> decoupled from internal details of HA implementation within Flume.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
