This type of error means that some other process/thread is listening on that port. You can use lsof and kill that process.
On 2022/04/27 18:52:33 "Le, Binh T." wrote: > We also just encountered this same issue as originally reported by Raivis. > I'm not sure what is going on either. It just happened all of a sudden when a > worker restarted. > > From: pwozniak <[email protected]> > Sent: Tuesday, February 15, 2022 8:28 AM > To: [email protected]; [email protected] > Subject: [External] Re: Address already bound issue > > This message is from an EXTERNAL SENDER - be CAUTIOUS, particularly with > links and attachments. > ________________________________ > > > Hi, > > we encountered similar problem lately. In our case it was related with JMX > port that we use for our worker. Quite strange in here is that it happened > some time after worker startup (not during worker startup). Our Storm version > is 2.3.0 (we lately upgraded it from 1.1.3). > > > Any Ideas would be appreciated :) > > 2022-01-26 19:43:09.655 e.e.c.s.d.s.AbstractDpsBolt > Thread-103-EDMObjectProcessorBolt-executor[17, 17] [INFO] Attempt number 2 to > process this message. No cleaning done here. > > > > 2022-01-27 09:43:35.558 STDERR Thread-0 [INFO] Error: JMX connector server > communication error: service:jmx:rmi://storm-supervisor2.novalocal:16700 > > 2022-01-27 09:43:35.583 STDERR Thread-0 [INFO] > jdk.internal.agent.AgentConfigurationError: java.rmi.server.ExportException: > Port already in use: 16700; nested exception is: > > 2022-01-27 09:43:35.584 STDERR Thread-0 [INFO] java.net.BindException: > Address already in use (Bind failed) > > 2022-01-27 09:43:35.584 STDERR Thread-0 [INFO] at > > jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:820) > > 2022-01-27 09:43:35.584 STDERR Thread-0 [INFO] at > > jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:479) > > 2022-01-27 09:43:35.585 STDERR Thread-0 [INFO] at > > jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:447) > > 2022-01-27 09:43:35.585 STDERR Thread-0 [INFO] at > > jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:599) > > 2022-01-27 09:43:35.585 STDERR Thread-0 [INFO] Caused by: > java.rmi.server.ExportException: Port already in use: 16700; nested exception > is: > > 2022-01-27 09:43:35.585 STDERR Thread-0 [INFO] java.net.BindException: > Address already in use (Bind failed) > > 2022-01-27 09:43:35.585 STDERR Thread-0 [INFO] at > > java.rmi/sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335) > > 2022-01-27 09:43:35.586 STDERR Thread-0 [INFO] at > > java.rmi/sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:243) > > 2022-01-27 09:43:35.586 STDERR Thread-0 [INFO] at > > java.rmi/sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412) > > 2022-01-27 09:43:35.586 STDERR Thread-0 [INFO] at > > java.rmi/sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) > > 2022-01-27 09:43:35.586 STDERR Thread-0 [INFO] at > > java.rmi/sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:234) > > 2022-01-27 09:43:35.586 STDERR Thread-0 [INFO] at > > jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap$PermanentExporter.exportObject(ConnectorBootstrap.java:203) > > 2022-01-27 09:43:35.587 STDERR Thread-0 [INFO] at > > java.management.rmi/javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:153) > > 2022-01-27 09:43:35.587 STDERR Thread-0 [INFO] at > > java.management.rmi/javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:138) > > 2022-01-27 09:43:35.587 STDERR Thread-0 [INFO] at > > java.management.rmi/javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:473) > > 2022-01-27 09:43:35.587 STDERR Thread-0 [INFO] at > > jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:816) > > 2022-01-27 09:43:35.588 STDERR Thread-0 [INFO] ... 3 more > > > > Regards, > > Pawel > > > On 12/22/21 11:16 AM, Raivis Strogonovs wrote: > Hello, > > I was wondering if anybody could help me out with the issue I'm having for > the past couple months. > > For some reason our storm instances are running fine for couple weeks, and > then suddenly it get's address bound issue, which can only be resolved by > restarting the respective machine. And it's not like it only happens one > machine, but randomly on any of them > > We are still using 2.2.0 at the moment. > The instances are running in docker. > > So far I haven't had luck at finding or figuring out, why would it happen... > > See logs here > > > 2021-12-22 10:03:39.057 o.a.s.m.n.Server main [INFO] Create Netty Server > Netty-server-localhost-6701, buffer_size: 5242880, maxWorkers: 1 > > 2021-12-22 10:03:39.062 STDERR Thread-0 [INFO] WARNING: An illegal reflective > access operation has occurred > > 2021-12-22 10:03:39.075 STDERR Thread-0 [INFO] WARNING: Illegal reflective > access by com.esotericsoftware.kryo.util.UnsafeUtil > (file:/apache-storm-2.2.0/lib-worker/kryo-3.0.3.jar<file://apache-storm-2.2.0/lib-worker/kryo-3.0.3.jar>) > to constructor java.nio.DirectByteBuffer(long,int,java.lang.Object) > > 2021-12-22 10:03:39.078 STDERR Thread-0 [INFO] WARNING: Please consider > reporting this to the maintainers of com.esotericsoftware.kryo.util.UnsafeUtil > > 2021-12-22 10:03:39.078 STDERR Thread-0 [INFO] WARNING: Use > --illegal-access=warn to enable warnings of further illegal reflective access > operations > > 2021-12-22 10:03:39.082 STDERR Thread-0 [INFO] WARNING: All illegal access > operations will be denied in a future release > > 2021-12-22 10:03:39.144 o.a.s.u.Utils main [ERROR] Received error in thread > main.. terminating server... > > java.lang.Error: java.security.PrivilegedActionException: > java.net.BindException: Address already in use > > at org.apache.storm.utils.Utils.handleUncaughtException(Utils.java:655) > ~[storm-client-2.2.0.jar:2.2.0] > > at org.apache.storm.utils.Utils.handleUncaughtException(Utils.java:634) > ~[storm-client-2.2.0.jar:2.2.0] > > at > org.apache.storm.utils.Utils.lambda$createDefaultUncaughtExceptionHandler$2(Utils.java:1016) > ~[storm-client-2.2.0.jar:2.2.0] > > at java.lang.ThreadGroup.uncaughtException(Unknown Source) [?:?] > > at java.lang.ThreadGroup.uncaughtException(Unknown Source) [?:?] > > at java.lang.Thread.dispatchUncaughtException(Unknown Source) [?:?] > > Caused by: java.security.PrivilegedActionException > > at java.security.AccessController.doPrivileged(Native Method) ~[?:?] > > at javax.security.auth.Subject.doAs(Unknown Source) ~[?:?] > > at org.apache.storm.daemon.worker.Worker.start(Worker.java:188) > ~[storm-client-2.2.0.jar:2.2.0] > > at org.apache.storm.daemon.worker.Worker.main(Worker.java:153) > ~[storm-client-2.2.0.jar:2.2.0] > > Caused by: java.net.BindException: Address already in use > > at sun.nio.ch.Net.bind0(Native Method) ~[?:?] > > at sun.nio.ch.Net.bind(Unknown Source) ~[?:?] > > at sun.nio.ch.Net.bind(Unknown Source) ~[?:?] > > at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) ~[?:?] > > at > org.apache.storm.shade.io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:130) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1358) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:1019) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:366) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at > org.apache.storm.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) > ~[storm-shaded-deps-2.2.0.jar:2.2.0] > > at java.lang.Thread.run(Unknown Source) ~[?:?] > > Any Ideas would be appreciated. > > > Thank You, > Raivis > > > Disclaimer: > > The content of this email is confidential and intended for the > recipient specified in message only. It is strictly forbidden to share any > part of this message with any third party, without a written consent of the > sender. If you received this message by mistake, please reply to this message > and follow with its deletion, so that we can ensure such a mistake does not > occur in the future. > The security of customers and suppliers as a priority. Therefore, we > have put efforts into ensuring that the message is error and virus-free. > Unfortunately, full security of the email cannot be ensured as, despite our > efforts, the data included in emails could be infected, intercepted, or > corrupted. Therefore, the recipient should check the email for threats with > proper software, as the sender does not accept liability for any damage > inflicted by viewing the content of this email. > No employee has the authority to conclude any binding contract > without an explicit written consent of their supervisor. Therefore, any will > to enter into an agreement must be confirmed by a second manager or director. > Any quotation contained is sent to compare available offers and > does not imply entering into a legally binding contract. > Please do not print this email unless it is necessary. Every > unprinted email helps the environment. > The views and opinions included in this email belong to their > author and do not necessarily mirror the views and opinions of the company. > Our employees are obliged not to make any defamatory clauses, infringe, or > authorise infringement of any legal right. Therefore, the company will not > take any liability for such statements included in emails. In case of any > damages or other liabilities arising, employees are fully responsible for the > content of their emails. > > ________________________________ > > This message is for the designated recipient only and may contain privileged, > proprietary, or otherwise confidential information. If you have received it > in error, please notify the sender immediately and delete the original. Any > other use of the e-mail by you is prohibited. Where allowed by local law, > electronic communications with Accenture and its affiliates, including e-mail > and instant messaging (including content), may be scanned by our systems for > the purposes of information security and assessment of internal compliance > with Accenture policy. Your privacy is important to us. Accenture uses your > personal data only in compliance with data protection laws. For further > information on how Accenture processes your personal data, please see our > privacy statement at https://www.accenture.com/us-en/privacy-policy. > ______________________________________________________________________________________ > > www.accenture.com >
