[
https://issues.apache.org/jira/browse/FLUME-989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227720#comment-13227720
]
[email protected] commented on FLUME-989:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4047/#review5841
-----------------------------------------------------------
Thanks for the patch Mike. I have one suggestion though:
Please modify the default appendBatch() method in the NettyAvroRpcClient()
implementation so that it does not throw an IllegalArgumentException if the
passed in collection contains more events than the allowed batch size. Instead
it should loop over the given collection in chunks of batch sizes until all of
it has been drained to the downstream end point.
Reason for this ask is that the builder can potentially use the built in
default value of batch size that is not visible to the client directly.
- Arvind
On 2012-03-09 23:44:00, Mike Percy wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4047/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-09 23:44:00)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Seeking early feedback on some additional APIs to make integrating with
Flume 1.x easier.
bq.
bq. Added the following APIs:
bq. - AvroClient: Friendly Java interface around the Avro API
bq. - AvroClientBuilder: Builder class to allow easy extension of AvroClient
capabilities in the future (i.e. SSL)
bq. - DefaultAvroClient: Implementation of the AvroClient interface
bq.
bq. Created this stuff in a flume-ng-sdk Maven submodule and moved the Event
interface to that submodule. flume-ng-core depends on flume-ng-sdk.
bq.
bq. I also modified AvroSink to use the AvroClient API instead of the bare
AvroSourceProtocol API directly.
bq.
bq.
bq. This addresses bug FLUME-989.
bq. https://issues.apache.org/jira/browse/FLUME-989
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-clients/flume-ng-log4jappender/pom.xml 9dd31bd
bq.
flume-ng-clients/flume-ng-log4jappender/src/main/java/org/apache/flume/clients/log4jappender/Log4jAppender.java
97f2b9e
bq. flume-ng-core/pom.xml fe6ce0b
bq. flume-ng-core/src/main/avro/flume.avdl 40da3ef
bq. flume-ng-core/src/main/java/org/apache/flume/Event.java 5278fc0
bq. flume-ng-core/src/main/java/org/apache/flume/EventDeliveryException.java
1413223
bq. flume-ng-core/src/main/java/org/apache/flume/FlumeException.java eab5b3d
bq.
flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java
195ba79
bq. flume-ng-core/src/main/java/org/apache/flume/event/EventBuilder.java
5d8c3b3
bq. flume-ng-core/src/main/java/org/apache/flume/event/SimpleEvent.java
e0c3b45
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 7386d06
bq. flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
a3f6640
bq. flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java
467785f
bq. flume-ng-core/src/test/java/org/apache/flume/util/TestEventBuilder.java
7930607
bq. flume-ng-sdk/pom.xml PRE-CREATION
bq. flume-ng-sdk/src/main/avro/flume.avdl PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/Event.java PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/EventDeliveryException.java
PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/FlumeException.java
PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClient.java
PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientFactory.java
PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/event/EventBuilder.java
PRE-CREATION
bq. flume-ng-sdk/src/main/java/org/apache/flume/event/SimpleEvent.java
PRE-CREATION
bq. flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java
PRE-CREATION
bq.
flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
PRE-CREATION
bq.
flume-ng-sdk/src/test/java/org/apache/flume/api/TestRpcClientFactory.java
PRE-CREATION
bq. flume-ng-sdk/src/test/java/org/apache/flume/event/TestEventBuilder.java
PRE-CREATION
bq. pom.xml d785762
bq.
bq. Diff: https://reviews.apache.org/r/4047/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Mike
bq.
bq.
> Factor Flume Avro RPC interfaces out into separate Client SDK
> -------------------------------------------------------------
>
> Key: FLUME-989
> URL: https://issues.apache.org/jira/browse/FLUME-989
> Project: Flume
> Issue Type: Sub-task
> Affects Versions: v1.0.0
> Reporter: Mike Percy
> Assignee: Mike Percy
> Fix For: v1.1.0
>
>
> Factor out the network RPC APIs so that there is a concise API boundary
> between user and developer APIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
