Yes, we might loose messages on the wire and I tried to address it in my previous version (that did not work). However, the ChannelFuture.success was not behaving as "expected". I need to go back to the board and figure the right way to do it. I can create a JIRA for that.
Regarding the parameters, I would let experienced people like Kishore, Bruce and you pick the default values - I ll make them tunable along with the above change. Thanks Karthik On Wed, Nov 30, 2011 at 9:57 AM, Matthieu Morel (Commented) (JIRA) < [email protected]> wrote: > > [ > https://issues.apache.org/jira/browse/S4-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160075#comment-13160075] > > Matthieu Morel commented on S4-7: > --------------------------------- > > I understand the pull request > https://github.com/leoneu/s4-piper/pull/10addresses this issue. > > I had a look at it: it looks quite nice, thanks Karthik! > > But I was wondering if you could clarify the following, before we can > merge that code: > > - in case of a network glitch, we lose messages "on the wire", right? I'm > not sure this is a requirement though... is it? (And it may introduce some > undesired overhead to handle that case). > > Minor thing: > - we'll have to provide a way to tune parameters instead of relying on > magic numbers (buffer size, number of retries), but that can be done later. > > > > Netty to tolerate network glitches and connection loss > > ------------------------------------------------------ > > > > Key: S4-7 > > URL: https://issues.apache.org/jira/browse/S4-7 > > Project: Apache S4 > > Issue Type: Bug > > Reporter: Leo Neumeyer > > Assignee: Karthik Kambatla > > Fix For: 0.5 > > > > > > NettyEmitter connects to different partitions and creates channels over > which it communicates to other listeners. > > It suffers from the following issues -- > > 1. If the underlying topology changes, the channels and the associated > connections are not updated. > > 2. If a connection gets disconnected, it stays disconnected. > > 3. If for any reason, a connection can't be made, send() drops the > message to be sent. > > The solution is to - > > 1. Maintain a bounded messageQueue for each destination partition - if a > connection does not exist, the message should be queued. > > 2. Maintain a map of the channel used for each destination partition - > update this map on changes to topology, or on send() in case of > disconnections. > > 3. Every time a (re-)connection is made, send the queued messages first. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators: > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > > >
