[
https://issues.apache.org/jira/browse/FLUME-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
simonchi updated FLUME-2688:
----------------------------
Priority: Blocker (was: Major)
> the avro sink in sinkgroup will never close the connection when process failed
> ------------------------------------------------------------------------------
>
> Key: FLUME-2688
> URL: https://issues.apache.org/jira/browse/FLUME-2688
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.5.2
> Environment: flume 1.5.2 two flume client and two flume
> ha(load_balance-->round_bin)
> Reporter: simonchi
> Priority: Blocker
>
> 169,177 two app server with flume agent to collect log and send them to
> remote ha flume by avro sink to connect with avro source.
> 84,88 two ha flume to receive the log from 169,177 by avro source to connect
> now when the time going on ,the connection between app and ha flume become
> more and more,it should be two connection in every ha flume ,ri ght ?
> but actually,it is not, as follows:
> [root@usercenter ~]# netstat -ano | grep 9520
> tcp 0 0 :::9520 :::*
> LISTEN off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:26371
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:43354
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:44055
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:18355
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:19968
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:21389
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:39635
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:56417
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:26416
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:63654
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:28258
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:29226
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:51033
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:21432
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:25367
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:16682
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:22288
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:23637
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:12397
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:40406
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:48131
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:39718
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:26650
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:32709
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:13135
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:28231
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:14285
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:19528
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:29502
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:46403
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:55529
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:32832
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:14920
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:30913
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:55971
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:46014
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:29924
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:60584
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:51096
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:14947
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:27936
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:33280
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:20794
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:16101
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:49969
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:45663
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:18501
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:63662
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:18487
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:34131
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:49777
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:26972
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:24546
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:48143
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:59868
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:26420
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:39200
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:21346
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.169:29591
> ESTABLISHED off (0.00/0/0)
> tcp 0 0 ::ffff:172.30.129.84:9520 ::ffff:172.30.128.177:35161
> ESTABLISHED off (0.00/0/0)
> so ,I think ,in the source code
> while (sinkIterator.hasNext()) {
> Sink sink = sinkIterator.next();
> try {
> status = sink.process();
> break;
> } catch (Exception ex) {
> selector.informSinkFailed(sink);
> LOGGER.warn("Sink failed to consume event. "
> + "Attempting next sink if available.", ex);
> }
> }
> in catch block,you should add a logic :
> if the sink is avro sink,you should close the old connection when it create a
> new connection.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
