Hello,
Problem is:
- Started (storm jar ...) DRPC topology with ShellBolt (PHP) (ver
9.3.0-rc1)
- After being started it works ok for us
- After being killed there is a garbage left somewhere according logs
(see logs below)
What can it be? In what direction should i continue my investigations?
Any ideas?
If you use(d) PHP for multilang bolts, could you please share Storm
version that you use and additional patches and hint that you use?
Thank you in advance,
Oleg
*** Logs ***
*** Nimbus (Seems "INFO Cleaning up domain" is continuing forever) ***
2014-11-12T21:11:14,387+01:00 host.name b.s.d.nimbus INFO Cleaning up
domain-check-1-1415820694
2014-11-12T21:11:24,479+01:00 host.name b.s.d.nimbus INFO Cleaning up
domain-check-1-1415820694
2014-11-12T21:11:34,560+01:00 host.name b.s.d.nimbus INFO Cleaning up
domain-check-1-1415820694
2014-11-12T21:11:34,658+01:00 host.name b.s.cluster WARN Could not
teardown heartbeats for domain-check-1-1415820694 #<RuntimeException
java.lang.RuntimeException:
org.apache.storm.zookeeper.KeeperException$NotEmptyException:
KeeperErrorCode = Directory not empty for
/workerbeats/domain-check-1-1415820694>
2014-11-12T21:11:44,698+01:00 host.name b.s.d.nimbus INFO Cleaning up
domain-check-1-1415820694
2014-11-12T21:11:54,778+01:00 host.name b.s.d.nimbus INFO Cleaning up
domain-check-1-1415820694
*** Supervisor node (this part repeated infinitly in the log as i can
see) ***
2014-11-13T08:20:56,678+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014
09:09 GMT
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:host.name=<WORKER1 IP>
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:java.version=1.7.0_71
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:java.vendor=Oracle Corporation
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client
environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71.x86_64/jre
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client
environment:java.class.path=/opt/storm-0.9.3/lib/clout-1.0.1.jar:/opt/storm-0.9.3/lib/logback-classic-1.0.13.jar:/opt/storm-0.9.3/lib/disruptor-3.2.1.jar:/opt/storm-0.9.3/lib/json-simple-1.1.jar:/opt/storm-0.9.3/lib/carbonite-1.4.0.jar:/opt/storm-0.9.3/lib/objenesis-1.2.jar:/opt/storm-0.9.3/lib/ring-devel-0.3.11.jar:/opt/storm-0.9.3/lib/commons-logging-1.1.3.jar:/opt/storm-0.9.3/lib/clojure-1.5.1.jar:/opt/storm-0.9.3/lib/clj-stacktrace-0.2.2.jar:/opt/storm-0.9.3/lib/core.incubator-0.1.0.jar:/opt/storm-0.9.3/lib/commons-exec-1.1.jar:/opt/storm-0.9.3/lib/compojure-1.1.3.jar:/opt/storm-0.9.3/lib/snakeyaml-1.11.jar:/opt/storm-0.9.3/lib/hiccup-0.3.6.jar:/opt/storm-0.9.3/lib/logback-core-1.0.13.jar:/opt/storm-0.9.3/lib/ring-core-1.1.5.jar:/opt/storm-0.9.3/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm-0.9.3/lib/asm-4.0.jar:/opt/storm-0.9.3/lib/clj-time-0.4.1.jar:/opt/storm-0.9.3/lib/math.numeric-tower-0.0.1.jar:/opt/storm-0.9.3/lib/tools.macro-0.1.0.jar:/opt/storm-0.9.3/lib/commons-io-2.4.jar:/opt/storm-0.9.3/lib/ring-servlet-0.3.11.jar:/opt/storm-0.9.3/lib/kryo-2.21.jar:/opt/storm-0.9.3/lib/jetty-6.1.26.jar:/opt/storm-0.9.3/lib/servlet-api-2.5.jar:/opt/storm-0.9.3/lib/storm-core-0.9.3-rc1.jar:/opt/storm-0.9.3/lib/tools.logging-0.2.3.jar:/opt/storm-0.9.3/lib/jetty-util-6.1.26.jar:/opt/storm-0.9.3/lib/commons-fileupload-1.2.1.jar:/opt/storm-0.9.3/lib/jgrapht-core-0.9.0.jar:/opt/storm-0.9.3/lib/tools.cli-0.2.4.jar:/opt/storm-0.9.3/lib/minlog-1.2.jar:/opt/storm-0.9.3/lib/slf4j-api-1.7.5.jar:/opt/storm-0.9.3/lib/commons-codec-1.6.jar:/opt/storm-0.9.3/lib/jline-2.11.jar:/opt/storm-0.9.3/lib/chill-java-0.3.5.jar:/opt/storm-0.9.3/lib/joda-time-2.0.jar:/opt/storm-0.9.3/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm-0.9.3/lib/reflectasm-1.07-shaded.jar:/opt/storm-0.9.3/lib/commons-lang-2.5.jar:/opt/storm-0.9.3/conf
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:java.io.tmpdir=/tmp
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:java.compiler=<NA>
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:os.name=Linux
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:os.arch=amd64
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:os.version=2.6.32-504.el6.x86_64
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:user.name=dev
2014-11-13T08:20:56,681+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:user.home=/opt/storm
2014-11-13T08:20:56,682+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Client environment:user.dir=/root
2014-11-13T08:20:56,693+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:host.name=<WORKER1 IP>
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:java.version=1.7.0_71
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:java.vendor=Oracle
Corporation
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server
environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71.x86_64/jre
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server
environment:java.class.path=/opt/storm-0.9.3/lib/clout-1.0.1.jar:/opt/storm-0.9.3/lib/logback-classic-1.0.13.jar:/opt/storm-0.9.3/lib/disruptor-3.2.1.jar:/opt/storm-0.9.3/lib/json-simple-1.1.jar:/opt/storm-0.9.3/lib/carbonite-1.4.0.jar:/opt/storm-0.9.3/lib/objenesis-1.2.jar:/opt/storm-0.9.3/lib/ring-devel-0.3.11.jar:/opt/storm-0.9.3/lib/commons-logging-1.1.3.jar:/opt/storm-0.9.3/lib/clojure-1.5.1.jar:/opt/storm-0.9.3/lib/clj-stacktrace-0.2.2.jar:/opt/storm-0.9.3/lib/core.incubator-0.1.0.jar:/opt/storm-0.9.3/lib/commons-exec-1.1.jar:/opt/storm-0.9.3/lib/compojure-1.1.3.jar:/opt/storm-0.9.3/lib/snakeyaml-1.11.jar:/opt/storm-0.9.3/lib/hiccup-0.3.6.jar:/opt/storm-0.9.3/lib/logback-core-1.0.13.jar:/opt/storm-0.9.3/lib/ring-core-1.1.5.jar:/opt/storm-0.9.3/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm-0.9.3/lib/asm-4.0.jar:/opt/storm-0.9.3/lib/clj-time-0.4.1.jar:/opt/storm-0.9.3/lib/math.numeric-tower-0.0.1.jar:/opt/storm-0.9.3/lib/tools.macro-0.1.0.jar:/opt/storm-0.9.3/lib/commons-io-2.4.jar:/opt/storm-0.9.3/lib/ring-servlet-0.3.11.jar:/opt/storm-0.9.3/lib/kryo-2.21.jar:/opt/storm-0.9.3/lib/jetty-6.1.26.jar:/opt/storm-0.9.3/lib/servlet-api-2.5.jar:/opt/storm-0.9.3/lib/storm-core-0.9.3-rc1.jar:/opt/storm-0.9.3/lib/tools.logging-0.2.3.jar:/opt/storm-0.9.3/lib/jetty-util-6.1.26.jar:/opt/storm-0.9.3/lib/commons-fileupload-1.2.1.jar:/opt/storm-0.9.3/lib/jgrapht-core-0.9.0.jar:/opt/storm-0.9.3/lib/tools.cli-0.2.4.jar:/opt/storm-0.9.3/lib/minlog-1.2.jar:/opt/storm-0.9.3/lib/slf4j-api-1.7.5.jar:/opt/storm-0.9.3/lib/commons-codec-1.6.jar:/opt/storm-0.9.3/lib/jline-2.11.jar:/opt/storm-0.9.3/lib/chill-java-0.3.5.jar:/opt/storm-0.9.3/lib/joda-time-2.0.jar:/opt/storm-0.9.3/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm-0.9.3/lib/reflectasm-1.07-shaded.jar:/opt/storm-0.9.3/lib/commons-lang-2.5.jar:/opt/storm-0.9.3/conf
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server
environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:java.io.tmpdir=/tmp
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:java.compiler=<NA>
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:os.name=Linux
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:os.arch=amd64
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server
environment:os.version=2.6.32-504.el6.x86_64
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:user.name=dev
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:user.home=/opt/storm
2014-11-13T08:20:56,694+01:00 <WORKER1 HOSTNAME>
o.a.s.z.s.ZooKeeperServer INFO Server environment:user.dir=/root
2014-11-13T08:20:57,213+01:00 <WORKER1 HOSTNAME> b.s.d.supervisor INFO
Starting Supervisor with conf {"dev.zookeeper.path"
"/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
"topology.skip.missing.kryo.registrations" false,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
"-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign"
true, "topology.trident.batch.emit.interval.millis" 500, "drpc.servers"
["<WORKER1 IP>" "185.27.173.47" "<WORKER3 IP>"],
"storm.messaging.netty.flush.check.interval.ms" 10,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/opt/storm", "storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true,
"nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs"
3600, "drpc.worker.threads" 64, "storm.meta.serialization.delegate"
"backtype.storm.serialization.DefaultSerializationDelegate",
"topology.worker.shared.thread.pool.size" 4, "nimbus.host"
"185.27.174.45", "storm.messaging.netty.min_wait_ms" 300,
"storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil,
"topology.executor.receive.buffer.size" 1024,
"transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm",
"storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads"
1, "storm.zookeeper.servers" ["<WORKER1 IP>" "185.27.173.47" "<WORKER3
IP>"], "transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128,
"worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port"
3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts"
"-Xmx768m", "topology.receiver.buffer.size" 8,
"task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
"storm.messaging.netty.max_retries" 300, "topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1,
"nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator",
"supervisor.slots.ports" [6700 6701 6702 6703 6704 6705 6706 6707 6708
6709], "topology.environment" nil, "topology.debug" false,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m",
"nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 3, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer"
"backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs"
30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count"
1, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 15, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport"
"backtype.storm.messaging.netty.Context", "logviewer.appender.name"
"A1", "storm.messaging.netty.max_wait_ms" 1500,
"drpc.request.timeout.secs" 600, "storm.local.hostname" "<WORKER1
HOSTNAME>", "storm.local.mode.zmq" false, "ui.port" 8080,
"nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144, "topology.classpath"
nil}
2014-11-13T08:20:57,244+01:00 <WORKER1 HOSTNAME>
b.s.u.StormBoundedExponentialBackoffRetry INFO The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
2014-11-13T08:20:57,325+01:00 <WORKER1 HOSTNAME>
o.a.s.c.f.i.CuratorFrameworkImpl INFO Starting
2014-11-13T08:20:57,328+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Initiating client connection, connectString=<WORKER1
IP>:2181,185.27.173.47:2181,<WORKER3 IP>:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@62c97092
2014-11-13T08:20:57,349+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Opening socket connection to server <WORKER3 IP>/<WORKER3 IP>:2181. Will
not attempt to authenticate using SASL (unknown error)
2014-11-13T08:20:57,356+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Socket connection established to <WORKER3 IP>/<WORKER3 IP>:2181,
initiating session
2014-11-13T08:20:57,406+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Session establishment complete on server <WORKER3 IP>/<WORKER3 IP>:2181,
sessionid = 0x349a7ffa2c50018, negotiated timeout = 20000
2014-11-13T08:20:57,409+01:00 <WORKER1 HOSTNAME>
o.a.s.c.f.s.ConnectionStateManager INFO State change: CONNECTED
2014-11-13T08:20:57,412+01:00 <WORKER1 HOSTNAME> b.s.zookeeper INFO
Zookeeper state update: :connected:none
2014-11-13T08:20:58,450+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Session: 0x349a7ffa2c50018 closed
2014-11-13T08:20:58,450+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
EventThread shut down
2014-11-13T08:20:58,452+01:00 <WORKER1 HOSTNAME>
b.s.u.StormBoundedExponentialBackoffRetry INFO The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
2014-11-13T08:20:58,453+01:00 <WORKER1 HOSTNAME>
o.a.s.c.f.i.CuratorFrameworkImpl INFO Starting
2014-11-13T08:20:57,328+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Initiating client connection, connectString=<WORKER1
IP>:2181,185.27.173.47:2181,<WORKER3 IP>:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@62c97092
2014-11-13T08:20:57,349+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Opening socket connection to server <WORKER3 IP>/<WORKER3 IP>:2181. Will
not attempt to authenticate using SASL (unknown error)
2014-11-13T08:20:57,356+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Socket connection established to <WORKER3 IP>/<WORKER3 IP>:2181,
initiating session
2014-11-13T08:20:57,406+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Session establishment complete on server <WORKER3 IP>/<WORKER3 IP>:2181,
sessionid = 0x349a7ffa2c50018, negotiated timeout = 20000
2014-11-13T08:20:57,409+01:00 <WORKER1 HOSTNAME>
o.a.s.c.f.s.ConnectionStateManager INFO State change: CONNECTED
2014-11-13T08:20:57,412+01:00 <WORKER1 HOSTNAME> b.s.zookeeper INFO
Zookeeper state update: :connected:none
2014-11-13T08:20:58,450+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Session: 0x349a7ffa2c50018 closed
2014-11-13T08:20:58,450+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
EventThread shut down
2014-11-13T08:20:58,452+01:00 <WORKER1 HOSTNAME>
b.s.u.StormBoundedExponentialBackoffRetry INFO The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
2014-11-13T08:20:58,453+01:00 <WORKER1 HOSTNAME>
o.a.s.c.f.i.CuratorFrameworkImpl INFO Starting
2014-11-13T08:20:58,453+01:00 <WORKER1 HOSTNAME> o.a.s.z.ZooKeeper INFO
Initiating client connection, connectString=<WORKER1
IP>:2181,185.27.173.47:2181,<WORKER3 IP>:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@62b51eb3
2014-11-13T08:20:58,454+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Opening socket connection to server <WORKER1 IP>/<WORKER1 IP>:2181. Will
not attempt to authenticate using SASL (unknown error)
2014-11-13T08:20:58,455+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Socket connection established to <WORKER1 IP>/<WORKER1 IP>:2181,
initiating session
2014-11-13T08:20:58,475+01:00 <WORKER1 HOSTNAME> o.a.s.z.ClientCnxn INFO
Session establishment complete on server <WORKER1 IP>/<WORKER1 IP>:2181,
sessionid = 0x149a7ffaa000014, negotiated timeout = 20000
2014-11-13T08:20:58,476+01:00 <WORKER1 HOSTNAME>
o.a.s.c.f.s.ConnectionStateManager INFO State change: CONNECTED
2014-11-13T08:20:58,589+01:00 <WORKER1 HOSTNAME> b.s.d.supervisor INFO
Starting supervisor with id 051ef04a-445e-402f-8010-f49519d363b0 at host
<WORKER1 HOSTNAME>
2014-11-13T08:20:59,358+01:00 <WORKER1 HOSTNAME> b.s.d.supervisor INFO
Shutting down and clearing state for id
79864941-ebae-4624-80bf-f53239440655. Current supervisor time:
1415863259. State: :disallowed, Heartbeat:
#backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1415863257,
:storm-id "domain-check-1-1415862954", :executors #{[2 2] [8 8] [14 14]
[-1 -1]}, :port 6708}
2014-11-13T08:20:59,358+01:00 <WORKER1 HOSTNAME> b.s.d.supervisor INFO
Shutting down
051ef04a-445e-402f-8010-f49519d363b0:79864941-ebae-4624-80bf-f53239440655
2014-11-13T08:20:59,370+01:00 <WORKER1 HOSTNAME> b.s.event ERROR Error
when processing event
java.io.IOException: . doesn't exist.
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:157) ~[commons-exec-1.1.jar:1.1]
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147) ~[commons-exec-1.1.jar:1.1]
at backtype.storm.util$exec_command_BANG_.invoke(util.clj:380)
~[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at
backtype.storm.util$send_signal_to_process.invoke(util.clj:409)
~[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at
backtype.storm.util$kill_process_with_sig_term.invoke(util.clj:420)
~[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at
backtype.storm.daemon.supervisor$shutdown_worker.invoke(supervisor.clj:194)
~[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at
backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:263)
~[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at clojure.lang.AFn.applyToHelper(AFn.java:161)
[clojure-1.5.1.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
at clojure.core$partial$fn__4190.doInvoke(core.clj:2396)
~[clojure-1.5.1.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:397)
~[clojure-1.5.1.jar:na]
at
backtype.storm.event$event_manager$fn__755.invoke(event.clj:40)
~[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
2014-11-13T08:20:59,381+01:00 <WORKER1 HOSTNAME> b.s.util ERROR Halting
process: ("Error when processing an event")
java.lang.RuntimeException: ("Error when processing an event")
at
backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:319)
[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at clojure.lang.RestFn.invoke(RestFn.java:423)
[clojure-1.5.1.jar:na]
at
backtype.storm.event$event_manager$fn__755.invoke(event.clj:48)
[storm-core-0.9.3-rc1.jar:0.9.3-rc1]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
2014-11-13T08:20:59,384+01:00 <WORKER1 HOSTNAME> b.s.d.supervisor INFO
Shutting down supervisor 051ef04a-445e-402f-8010-f49519d363b0
2014-11-13T08:20:59,386+01:00 <WORKER1 HOSTNAME> b.s.event INFO Event
manager interrupted
