i used netstat -ntape
and got Code: tcp 0 0 0.0.0.0:6703 0.0.0.0:* LISTEN 1000 22776 2621/java then used this command with PID of PORT = 2621 Code: lsof -P -w -n -p 2621 -a -i and got Code: COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 2621 user 76u IPv4 18906 0t0 TCP 127.0.0.1:50036->127.0.0.1:2181 (ESTABLISHED) java 2621 user 83u IPv4 22773 0t0 TCP 127.0.0.1:56733->127.0.0.1:3773 (ESTABLISHED) java 2621 user 86u IPv4 22776 0t0 TCP *:6703 (LISTEN) On Fri, Dec 11, 2015 at 4:44 AM, 刘键(Basti Liu) <[email protected]> wrote: > Hi Sam, > > Have you tried "netstat" and "lsof" commands recommended by Erik? What is > the result? > We need to find which process bound this port, then kill the process to > release this port. > > Regards > Basti > > -----Original Message----- > From: sam mohel [mailto:[email protected]] > Sent: Friday, December 11, 2015 8:15 AM > To: [email protected] > Subject: Re: Problem with storm since 4 months > > > now i converted the change > > but problem still > > > i mean in the link you sent , he said "the ephermal port range is > specified in /proc/sys/net/ipv4/ip_local_ port_range. You can probably > extend it to run from 16k to 64k." > > how can i do that ? > > > how can i check if there is anothe dump worker working , > or > is there another way to kill any worker if it is dump or not "completely" > and start from scratch > > thanks for your patience and your time > > > On Thu, Dec 10, 2015 at 7:38 PM, Erik Weathers < > [email protected]> wrote: > > > responses inline. > > > > >Ah, so, you shouldn't have touched the "local port range" setting > > >since > > you > > don't fully understand ephemeral ports and TCP yet! :-) > > > > I'll give a brief synopsis: > > > > Say you're making a connection from a client to a server that listens > > on port 80. The client needs to have a port of its own to receive the > > response packets from the server. The port that is allocated for it by > > the TCP stack is a "random" port, which is called an "ephemeral" port in > this context. > > So with your original default config (32768 61000) the port would be > > chosen from an available port on the machine that lies within that > > range of > > 32768->61000. That's good, it wouldn't conflict with the default TCP > > 32768->ports > > being listened to by the Storm Worker processes (67xx). So 1. isn't > > your problem. But now you've made it possible for it to become a > > problem, since now the range of ports that can be given as an > > ephemeral port overlaps with the default Storm Worker ports > > (1024->65535 includes 67xx). So you should revert that config change. > > > > Thanks for your replying , now i reverted the change > > > > > >Sorry, I don't understand what you are asking. What thing is "16" > > >that > > you > > are trying to extend? > > > > i mean in the link you sent , he adviced to extend from 16 to 64 > > >You can kill topologies from the Nimbus UI (web page). Or with the > > bin/storm command. > > > > > > >The commands are *solely* intended to figure out what is conflicting. > > Your > > logs claim that there is something holding onto 67xx which prevents the > > Storm Worker from launching. So if that is happening you should > > *immediately* try to figure out what is actually holding onto the port > > and preventing your Storm Worker from launching. > > > > > > >Not sure what you mean here. > > > > Please note that Storm often suffers from "cascading" failures, where > > there are a lot of exceptions and errors that aren't actually the root > > cause of the problem. Often you need to spend time and effort looking > > at lots of logs and tracing back to the real root cause. > > > > > > > > > > > > > > Really , Thanks for your time > > > > > > On Thu, Dec 10, 2015 at 6:22 AM, Erik Weathers < > > > [email protected]> wrote: > > > > > > > Regarding Basti's suggestion (1.) that your host's configured > > > > ephemeral ports might be conflicting with the storm worker ports, > > > > here's how you > > > can > > > > check your "local port range" setting: > > > > > > > > > > > > > > > > > https://serverfault.com/questions/261663/on-linux-how-can-i-tell-how-m > > any-ephemeral-ports-are-left-available > > > > > > > > % cat /proc/sys/net/ipv4/ip_local_port_range > > > > > > > > > > > > It's possible that there is a zombie worker process holding onto > > > > port > > > 6703. > > > > I would try to identify the process like so: > > > > > > > > % sudo netstat -ap --numeric-ports --extend | grep -w LISTEN | > > > > grep -w > > > 6703 > > > > > > > > Alternatively you can try a global lsof search: > > > > > > > > % sudo lsof | grep TCP | grep -w LISTEN | grep -w 6703 > > > > > > > > - Erik > > > > > > > > > > > > > > > > On Wed, Dec 9, 2015 at 7:37 PM, 刘键(Basti Liu) < > > [email protected]> > > > > wrote: > > > > > > > > > Hi Sam, > > > > > > > > > > You can try to find which process has bound this port by > > > > > "netstat > > -anp" > > > > > first. > > > > > > > > > > Generally, there are following cases for the binding error. > > > > > 1. "local port range" is not set to exclude the port range used > > > > > in > > > Storm. > > > > > 2. The previous worker was not killed correctly. > > > > > 3. There is bug of assignment in some scenarios. Same port was > > assigned > > > > to > > > > > two workers. > > > > > > > > > > Regards > > > > > Basti > > > > > > > > > > -----Original Message----- > > > > > From: sam mohel [mailto:[email protected]] > > > > > Sent: Thursday, December 10, 2015 7:16 AM > > > > > To: [email protected] > > > > > Subject: Re: Problem with storm since 4 months > > > > > > > > > > i tried to use storm-0.9.5 but problem changed with > > > > > > > > > > cannot bind port 6703 i think it's same problem > > > > > > > > > > On Wed, Dec 9, 2015 at 8:42 PM, Harsha <[email protected]> wrote: > > > > > > > > > > > Sam, > > > > > > you might be using very old version of storm since > > > > > > its > > > > showing > > > > > > ZeroMQ. Can you try using newer version storm > > > > > > without > > zero > > > > mq. > > > > > > -Harsha > > > > > > > > > > > > On Wed, Dec 9, 2015, at 10:19 AM, sam mohel wrote: > > > > > > > I have this problem since 4months when I submitted topology > > > > > > > I got this in the worker log file [ERROR] Async loop died! > > > > > org.zeromq.ZMQException: > > > > > > > Address already in use(0x62) at > > > > > > > org.zeromq.ZMQ$Socket.bind(Native Method) at > > > > > > > zilch.mq$bind.invoke(mq.clj:69) at > > > > > > > backtype.storm.messaging.zmq.ZMQContext.bind(zmq.clj:57)at > > > > > > > > > > > > > > > > backtype.storm.messaging.loader$launch_receive_thread_BANG_$fn__1629 > > > .i > > > > > > nvoke(loader.clj:26) > > > > > > > at > > > > > > > backtype.storm.util$async_loop$fn__465.invoke(util.clj:375) > > > > > > > at clojure.lang.AFn.run(AFn.java:24) at > > > java.lang.Thread.run(Unknown > > > > > > > Source) > > > > > > > > > > > > > > when i tried to connect port 6703 and 6702 > > > > > > > > > > > > > > And supervisor log file hadn't still start > > > > > > > > > > > > > > > > > > > > > I searched everywhere but cannot find any solution I hope > > > > > > > you can > > > > > > > > > > > > > > > > > > > > > > > > > > >
