[
https://issues.apache.org/jira/browse/FLUME-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560329#comment-16560329
]
ASF GitHub Bot commented on FLUME-3262:
---------------------------------------
GitHub user JohnZZGithub opened a pull request:
https://github.com/apache/flume/pull/219
FLUME-3262: Improve Scribe Source With More Options
We added following options:
1. ThriftServerType, user choose one of {THshaServer,
TThreadedSelectorServer, TThreadPoolServer} baed on their own traffic pattern.
More details on thrift server could be found on Apache Thrift project.
2. MaxThriftFrameSizeBytes: thrift frame size limit.
3. MaxReadBufferSize. The buffer size thrift used to handle requests,
increase the buffer will benefit throughput under heavy load.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JohnZZGithub/flume FLUME-3262
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/219.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #219
----
commit c8277f0ee02023c51c882d472622dd2b17be2b9b
Author: JohnZZGithub <johnzzgithub@...>
Date: 2018-07-27T21:04:59Z
FLUME-3262: Improve Scribe Source With More Options
We added following options:
1. ThriftServerType, user choose one of {THshaServer,
TThreadedSelectorServer, TThreadPoolServer} baed on their own traffic pattern.
More details on thrift server could be found on Apache Thrift project.
2. MaxThriftFrameSizeBytes: thrift frame size limit.
3. MaxReadBufferSize. The buffer size thrift used to handle requests,
increase the buffer will benefit throughput under heavy load.
----
> Improve Scribe Source With More Options
> ---------------------------------------
>
> Key: FLUME-3262
> URL: https://issues.apache.org/jira/browse/FLUME-3262
> Project: Flume
> Issue Type: New Feature
> Reporter: zhenzhao wang
> Priority: Major
> Attachments: FLUME-3262-0.patch
>
>
> We're a heavy scribe source user with tremendous traffic. We found there're
> limit configuration options in original scribe source. So we add more options
> including:
> # ThriftServerType, user choose one of {THshaServer, TThreadedSelectorServer,
> TThreadPoolServer} base their own traffic pattern. More details on thrift
> server could be found on Apache Thrift project.
> # maxThriftFrameSizeBytes, the default max thrift frame size is 16M, it
> should be good most times, however there might some cases that the thrift
> size is bigger than 16M. The thrift server will just throw exception without
> too many valid info when a msg is bigger than the threshold. Some user might
> prefer receive, print out some msg before discard it, or some user just have
> event size bigger than 16M
> # Max read buffer size. The buffer size thrift used to handle requests,
> increase the buffer will benefit throughput under heavy load.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
