Hi Asankha, this night I was able to consistently reproduce this issue and track it down. In order to gracefully restart an instance we do something like that:
shutdownGracefully(seconds); startup(); both method internally use the same MBeanServerConnection and invoke the respective commands. If we call this method for the first time we get the exception as reported. If we check the ports with lsof after the execution all http(s) listeners are gone. If we than issue the command for the second time, it succeeds. This is because the listeners had been shutdown before. The third call of the command is like the first. After that everything repeats... If we change the code to something silly like: shutdownGracefully(seconds); Thread.sleep(2000); startup(); it consistently works. So this is a timing issue. The MBean-Method shutdownGracefully returns, before the server sockets are closed. The strange thing is, that I was not able to reproduce this issue in other environments. Maybe this is because the machines in this environment are much faster. The whole cluster-reload takes less than a second without the sleep. Could you please check the code for a bug! We need this feature to work reliably. Thanks, Eric > Hi Eric > > While testing the graceful cluster restart (we internally call > > "reload-cluster" to differentiate from a "real" restart in our QA > > environment we encountered the following problem: > > ... > > > I cannot reproduce this at my end.. and I can shutdown gracefully and > restart any number of times without any issues.. > > 2008-06-09 10:55:49,391 [10.10.3.87-app40.jamba.easynet.de] > > [HttpCoreNIOListener] WARN HttpCoreNIOListener System may be unstable: > > IOReactor encountered a ch > > ecked exception : Address already in use > > java.net.BindException: Address already in use..... > > ....... > > org.apache.synapse.transport.nhttp.HttpCoreNIOListener.access$200(HttpCo > > reNIOListener.java:59) > > at > > org.apache.synapse.transport.nhttp.HttpCoreNIOListener$2.run(HttpCoreNIO > > Listener.java:273) > > at java.lang.Thread.run(Thread.java:595) > > 2008-06-09 10:55:49,392 [10.10.3.87-app40.jamba.easynet.de] [RMI TCP > > Connection(5)-10.10.3.87] INFO HttpCoreNIOListener HTTPS Listener > > starting on port : 118 > > 12 > > > > > This clearly seems to indicate that the http/s ports were still in use > when you were trying the "start" after the graceful shutdown. Can you do > a "netstat -na | grep LISTEN | grep tcp" after the graceful shutdown to > see if the ports are still being used? Can you also try the ESB 1.7 > vanilla installation and try a graceful shutdown followed by a restart.. > lets see whats different in your environment.. _______________________________________________ Esb-java-dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/esb-java-dev
