Is the hedwig hub closing the connection to the Bookies or are the bookies closing connections to the hedwig hubs? The bookie logs have messages like
2012-04-03 22:09:23,153 - DEBUG [NIOServerFactory-3181:NIOServerFactory$Cnxn@398] - close NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.35.80.102:3181 remote=/ 10.35.238.118:50022] When I check 10.35.238.118 (which is a hedwig hub that is not responsible for any topics), the logfile shows no disconnections. Only the logfile for the Hedwig hub responsible for the topic shows the disconnections to the bookies. I'm not able to differentiate based on time because the time entries in the logfile are exactly the same, down to the millisecond. On Tue, Apr 3, 2012 at 2:09 AM, Ivan Kelly <[email protected]> wrote: > This type of disconnection occurs when there's a read timeout from one of > the bookies. The cause could be something crashing on the bookie side, or > simply a very slow network. What type of network are you running this in? > Do you have any logs on the bookie side? > > -Ivan > > On 3 Apr 2012, at 03:22, Aniruddha Laud wrote: > > > While sending requests to a hedwig hub, the hub seems to disconnect from > > the bookies and never connects back. The logfile contains > > > > 2012-04-02 22:33:09,207 - INFO [Hashed wheel timer > > #3:PerChannelBookieClient@409] - Disconnected from bookie: / > > 10.35.84.103:3181 > > 2012-04-02 22:33:09,211 - INFO [Hashed wheel timer > > #4:PerChannelBookieClient@409] - Disconnected from bookie: / > > 10.34.133.114:3181 > > 2012-04-02 22:33:09,214 - INFO [Hashed wheel timer > > #5:PerChannelBookieClient@409] - Disconnected from bookie: / > > 10.35.89.103:3181 > > 2012-04-02 22:33:09,217 - INFO [Hashed wheel timer > > #8:PerChannelBookieClient@409] - Disconnected from bookie: / > > 10.35.91.102:3181 > > 2012-04-02 22:33:09,247 - INFO [Hashed wheel timer > > #10:PerChannelBookieClient@409] - Disconnected from bookie: / > > 10.34.234.125:3181 > > 2012-04-02 22:33:09,256 - INFO [Hashed wheel timer > > #7:PerChannelBookieClient@409] - Disconnected from bookie: / > > 10.34.235.129:3181 > > > > Some time before getting this message, the "Got response for ..." > messages > > stop and there are only "Successfully wrote request ..." messages in the > > hedwig log file. The bookkeeper log-file shows no indication of the > > connection being lost. All the bookies and hedwig hubs are up and running > > and I am able to connect to them with the hedwig console and able to > create > > new topics and publish/subscribe to them. But I'm not able to publish or > > subscribe to the topic that caused the errors. About 200,000 entries were > > created in the topic that caused this error. > > > > I'm unable to attach the log files or even portions of it, because the > > relevant portions are around 3MB. > > > > Regards, > > Aniruddha. > >
