During each topology kill/restart? Where do you put that hook? Or was it a one time remove of /tmp/storm and future topology restarts were fine.
Sent from my iPhone > On Mar 25, 2015, at 1:59 PM, Andres Gomez Ferrer <[email protected]> wrote: > > I solved it removing all /tmp/storm/* > > Regards, > > Andrés Gómez > Developer > redborder.net / [email protected] > Phone: +34 955 60 11 60 > > <0e6e8de_1.png> > > <square-twitter-20.png> <square-google-plus-20.png> <square-linkedin-20.png> > > Piénsalo antes de imprimir este mensaje > > Este correo electrónico, incluidos sus anexos, se dirige exclusivamente a su > destinatario. Contiene información CONFIDENCIAL cuya divulgación está > prohibida por la ley o puede estar sometida a secreto profesional. Si ha > recibido este mensaje por error, le rogamos nos lo comunique inmediatamente y > proceda a su destrucción. > > This email, including attachments, is intended exclusively for its addressee. > It contains information that is CONFIDENTIAL whose disclosure is prohibited > by law and may be covered by legal privilege. If you have received this email > in error, please notify the sender and delete it from your system. > > > En 25 de marzo de 2015 en 20:54:58, Justin Workman ([email protected]) > escrito: > >> We are currently running 7 node storm cluster, 1 nimbus and 6 supervisor >> nodes all running storm 0.9.2, running 3 topologies. Any time we kill a >> running topology the supervisors across all nodes start flapping and we end >> up in a mess. To clean this up we end up killing all running topologies, >> shutdown the supervisors, cleanup the storm/storm-local directories on all >> supervisor nodes, restart the supervisor processes then restart the >> topologies. >> >> Has anyone experienced this issues, or have any ideas on how to resolve it. >> >> Log snippet we see in the supervisor logs when this happens... >> >> 2015-03-25 11:28:13 b.s.d.supervisor [INFO] Shutting down >> 4d971d4b-a208-4758-a55e-3e8b34d7531f:ce049d5c-fd4c-499c-ad8d-ef1d8f2b992b >> 2015-03-25 11:28:13 b.s.event [ERROR] Error when processing event >> java.io.IOException: . doesn't exist. >> at >> org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:157) >> ~[commons-exec-1.1.jar:1.1] >> at >> org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147) >> ~[commons-exec-1.1.jar:1.1] >> at backtype.storm.util$exec_command_BANG_.invoke(util.clj:378) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at >> backtype.storm.util$ensure_process_killed_BANG_.invoke(util.clj:394) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at >> backtype.storm.daemon.supervisor$shutdown_worker.invoke(supervisor.clj:175) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at >> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:240) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at clojure.lang.AFn.applyToHelper(AFn.java:161) >> ~[clojure-1.5.1.jar:na] >> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.5.1.jar:na] >> at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na] >> at clojure.core$partial$fn__4190.doInvoke(core.clj:2396) >> ~[clojure-1.5.1.jar:na] >> at clojure.lang.RestFn.invoke(RestFn.java:397) >> ~[clojure-1.5.1.jar:na] >> at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.5.1.jar:na] >> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] >> 2015-03-25 11:28:13 b.s.util [INFO] Halting process: ("Error when processing >> an event") >> >> >> There does not be any thing corresponding to this in the worker logs.' >> >> Ideas?? >> >> Thanks >> Justin
