Sorry one remark that in order to fix it I don't need to delete anything from folders (but maybe I can) I need to just kill pid but its anyway java process and I am 100% sure its storm topology.
пт, 2 дек. 2022 г. в 18:25, Nikita <[email protected]>: > Hi, I have been facing Address already in use cycle error for a long > period of time. I saw the most common answer that it means that something > is listening on that port. But as I understood from logs after storm asks > for rebalancing (by some reason when storm gets a new leader assigned) or > you deploy\redeploy topologies it happens. But its not other process is > listening on port its storm topology listening on that port. Supervisor > launch a topology (for example 2:15:00) and after some time (based on first > heartbeat (at 2:15:05) change status of a slot from > waiting-for-worker-start to kill-blob-update by unknown for me reason and > starts cleaning paths from storm/workers/id and > storm/workers-users/topology_id, Removed Worker ID and so on and finished. > But after few seconds (for example at 2:15:10) worker created that > instance. And what we have, supervisor deleted that worker id and don’t > know already about the topology and thinks it was removed but actually > worker created that topology instance and all calls of a supervisor after > to create a topology fail because supervisor sees that port is free but > actually its not. (you can read full explanations by me, log samples, > config here > https://stackoverflow.com/questions/74345286/apache-storm-address-already-in-use-error > ). > I may be wrong but why I am sure it’s a topology is listening on port: > 1) I see a folder with worker id in storm/workers with creating date (and > last update) at +- deploying time. > 2) Same related to workers-artifacts\EventHandler-3-1669745351\6700 > (EventHandler-3-1669745351 it’s a name + id of a problem topology) > 3)I can fix it either by stopping storm, deleting storm/ folder and > cleaning zookeeper or just by removing that worker id folder manually on > supervisor machine and after that exception disappears. But its pretty > annoying and hard to find fast a folder, for prod it may be critical. > > I would be very thankful to you if you help me to solve that one of > annoying storm issues. I couldn’t find a conf to add some delay to it. > > additional info: > logs attached, busy port if 6700, EventHandler-3-1669745351 is a topology > name+id, worker id 1d375d63-38e3-4b6d-8d9e-4db7dd9bc20e > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows > > >
