Hi all,
I am currently trying use TestTopologyStaticHosts to try connect the KafkaSpout
to a Kafka topic. I have a ZooKeeper and a Kafka instance running on my
localhost. I have a topic named "twitter-topic" that has some tweets in it.
This is all working as expected. I can run the consumer in the terminal and it
returns the tweets. I want to use the KafkaSpout to connect to the Kafka topic
and pull the tweets into a topology. I have been working on this a few days now
and no success.
So far i have learned that when Storm is run in local mode that it uses an in
memory zookeeper on port 2000, which would not allow it to connect to the Kafka
topic. I have tried to get around this using the following syntax that i found
online:
LocalCluster cluster = new LocalCluster("localhost", new Long(2181));
It is still not working but it seems to be connecting to Kafka as it gives a
'closed socket connection' message when i cancel the operation (after it does
not work and hangs open). It also says in the storm output that it is connected
to localhost 2181 so it seems to be getting that far. I have included the full
output from Storm in a txt file attached.
Here is the code i am using in the TestTopologyStaticHosts class:
public static void main(String[] args) throws Exception {
//String zkConnString = "localhost:2181";
GlobalPartitionInformation hostsAndPartitions = new
GlobalPartitionInformation(); hostsAndPartitions.addPartition(0, new
Broker("127.0.0.1", 9092)); BrokerHosts brokerHosts = new
StaticHosts(hostsAndPartitions); // BrokerHosts brokerHosts = new
ZkHosts(zkConnString, "/brokers");
SpoutConfig kafkaConfig = new SpoutConfig(brokerHosts,
"twitter-topic","/twitter","twitter-topic-id"); kafkaConfig.scheme = new
SchemeAsMultiScheme(new StringScheme());
//kafkaConfig.forceStartOffsetTime(-2);
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("words", new KafkaSpout(kafkaConfig), 10);
builder.setBolt("print", new PrinterBolt()).shuffleGrouping("words");
LocalCluster cluster = new LocalCluster("localhost", new Long(2181));
Config config = new Config(); config.setDebug(true); //
config.put("storm.zookeeper.servers", "localhost"); //
config.put("storm.zookeeper.port", "2181");
cluster.submitTopology("kafka-test", config, builder.createTopology());
Thread.sleep(600000);
}
Judging by the output it seems that there is a problem with connecting to the
Kafka partitions.I have tried many different things to get it to work but no
luck. I have also been looking at using the KafkaSpoutTestTopology class but it
is expecting arguments including 'dockerIp' which i don't understand.
Should i be using Storm in localmode?Should i be using the
TestTopologyStaticHosts class or would the KafkaSpoutTestTopology class be
better?
Any help at all would be greatly appreciated because i am really stuck.
Kind RegardsDavid Kavanagh
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building storm-kafka 0.9.6
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-remote-resources-plugin:1.2.1:process (default) @ storm-kafka
---
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
storm-kafka ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/david_kav/Desktop/College/Project/storm-0.9.6/external/storm-kafka/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ storm-kafka ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 50 source files to
/Users/david_kav/Desktop/College/Project/storm-0.9.6/external/storm-kafka/target/classes
[WARNING]
/Users/david_kav/Desktop/College/Project/storm-0.9.6/external/storm-kafka/src/jvm/storm/kafka/KafkaUtils.java:
Some input files use unchecked or unsafe operations.
[WARNING]
/Users/david_kav/Desktop/College/Project/storm-0.9.6/external/storm-kafka/src/jvm/storm/kafka/KafkaUtils.java:
Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] >>> exec-maven-plugin:1.2.1:java (default-cli) > validate @ storm-kafka
>>>
[INFO]
[INFO] <<< exec-maven-plugin:1.2.1:java (default-cli) < validate @ storm-kafka
<<<
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ storm-kafka ---
3447 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
3452 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:host.name=192.168.1.11
3452 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:java.version=1.7.0_45
3452 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle
Corporation
3452 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
3452 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:java.class.path=/usr/local/Cellar/maven/3.3.9/libexec/boot/plexus-classworlds-2.5.2.jar
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:java.library.path=/Users/david_kav/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:java.io.tmpdir=/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T/
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:os.name=Mac OS X
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:os.arch=x86_64
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:os.version=10.11.4
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client environment:user.name=david_kav
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:user.home=/Users/david_kav
3453 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Client
environment:user.dir=/Users/david_kav/Desktop/College/Project/storm-0.9.6/external/storm-kafka
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:host.name=192.168.1.11
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.version=1.7.0_45
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.vendor=Oracle Corporation
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.class.path=/usr/local/Cellar/maven/3.3.9/libexec/boot/plexus-classworlds-2.5.2.jar
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.library.path=/Users/david_kav/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.io.tmpdir=/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T/
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:java.compiler=<NA>
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:os.name=Mac OS X
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:os.arch=x86_64
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:os.version=10.11.4
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:user.name=david_kav
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:user.home=/Users/david_kav
3469 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.server.ZooKeeperServer - Server
environment:user.dir=/Users/david_kav/Desktop/College/Project/storm-0.9.6/external/storm-kafka
4136 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.nimbus - Starting Nimbus with conf {"dev.zookeeper.path"
"/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs"
10, "logviewer.childopts" "-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size"
1024, "storm.local.dir"
"/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T//f11f892c-1a5c-4b06-8132-b2eca88dee75",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts"
true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs"
3600, "drpc.worker.threads" 64, "storm.meta.serialization.delegate"
"backtype.storm.serialization.DefaultSerializationDelegate",
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm",
"storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers"
("localhost"), "transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts"
"-Xmx768m", "supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 300,
"topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
[6700 6701 6702 6703], "topology.environment" nil, "topology.debug" false,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port"
6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
"topology.disruptor.wait.timeout.millis" 1000, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1,
"storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport" "backtype.storm.messaging.netty.Context",
"logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080,
"nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144, "topology.classpath" nil}
4145 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.nimbus - Using default scheduler
4157 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
4316 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
4319 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@60e54a13
4339 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
4345 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
4352 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/127.0.0.1:2181, sessionid = 0x153c1a14d4900a6, negotiated
timeout = 20000
4355 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
4356 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
backtype.storm.zookeeper - Zookeeper state update: :connected:none
5375 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.zookeeper.ClientCnxn - EventThread shut down
5375 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Session: 0x153c1a14d4900a6 closed
5377 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
5377 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
5378 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@442a559d
5379 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
5379 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
5381 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/127.0.0.1:2181, sessionid = 0x153c1a14d4900a7, negotiated
timeout = 20000
5381 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
5398 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.nimbus - Corrupt topology kafka-test-1-1459265609 has
state on zookeeper but doesn't have a local dir on Nimbus. Cleaning up...
5413 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
5413 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
5414 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@32cefcee
6178 [timer] INFO backtype.storm.daemon.nimbus - Cleaning up
kafka-test-1-1459265609
10483 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181. Will not attempt to
authenticate using SASL (unknown error)
10483 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, initiating session
10485 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, sessionid =
0x153c1a14d4900a8, negotiated timeout = 20000
10485 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
10486 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
backtype.storm.zookeeper - Zookeeper state update: :connected:none
10488 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Session: 0x153c1a14d4900a8 closed
10488 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.zookeeper.ClientCnxn - EventThread shut down
10488 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
10488 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
10489 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@7c9522c0
10490 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
10490 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
10491 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
10491 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
10491 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@44d8bd53
10494 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/127.0.0.1:2181, sessionid = 0x153c1a14d4900a9, negotiated
timeout = 20000
10495 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
15569 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181. Will not attempt to
authenticate using SASL (unknown error)
15569 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, initiating session
15571 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, sessionid =
0x153c1a14d4900aa, negotiated timeout = 20000
15571 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
15572 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
backtype.storm.zookeeper - Zookeeper state update: :connected:none
15573 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Session: 0x153c1a14d4900aa closed
15573 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.zookeeper.ClientCnxn - EventThread shut down
15574 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
15574 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
15574 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@16b85c5e
15576 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL
(unknown error)
15576 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/0:0:0:0:0:0:0:1:2181, initiating session
15577 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x153c1a14d4900ab,
negotiated timeout = 20000
15577 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
15596 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.supervisor - Starting Supervisor with conf
{"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs"
10, "logviewer.childopts" "-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size"
1024, "storm.local.dir"
"/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T//7a4ed588-4f64-4bee-a6f8-446eba77b696",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts"
true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs"
3600, "drpc.worker.threads" 64, "storm.meta.serialization.delegate"
"backtype.storm.serialization.DefaultSerializationDelegate",
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm",
"storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers"
("localhost"), "transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts"
"-Xmx768m", "supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 300,
"topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1024 1025 1026), "topology.environment" nil, "topology.debug" false,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port"
6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
"topology.disruptor.wait.timeout.millis" 1000, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1,
"storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport" "backtype.storm.messaging.netty.Context",
"logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080,
"nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144, "topology.classpath" nil}
15616 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
15617 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
15617 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@188d83fc
15619 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
15619 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
15620 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/127.0.0.1:2181, sessionid = 0x153c1a14d4900ac, negotiated
timeout = 20000
15678 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
15679 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
backtype.storm.zookeeper - Zookeeper state update: :connected:none
15681 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Session: 0x153c1a14d4900ac closed
15681 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.zookeeper.ClientCnxn - EventThread shut down
15681 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
15682 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
15682 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@7e4dff
20689 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181. Will not attempt to
authenticate using SASL (unknown error)
20690 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, initiating session
20692 [storm.kafka.TestTopologyStaticHosts.main(fe80:0:0:0:0:0:0:1%1:2181)]
INFO org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, sessionid =
0x153c1a14d4900ad, negotiated timeout = 20000
20692 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
20735 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.supervisor - Starting supervisor with id
b211e8c0-0b5b-4797-848c-abc7dc7bac6b at host 192.168.1.11
20739 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.supervisor - Starting Supervisor with conf
{"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs"
10, "logviewer.childopts" "-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size"
1024, "storm.local.dir"
"/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T//d8d541cb-a98f-44c3-b09f-5a23a22daa67",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts"
true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs"
3600, "drpc.worker.threads" 64, "storm.meta.serialization.delegate"
"backtype.storm.serialization.DefaultSerializationDelegate",
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm",
"storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers"
("localhost"), "transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts"
"-Xmx768m", "supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 300,
"topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1027 1028 1029), "topology.environment" nil, "topology.debug" false,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port"
6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
"topology.disruptor.wait.timeout.millis" 1000, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1,
"storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport" "backtype.storm.messaging.netty.Context",
"logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080,
"nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144, "topology.classpath" nil}
20787 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
20788 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
20788 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@60562a70
20789 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
20789 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
20790 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/127.0.0.1:2181, sessionid = 0x153c1a14d4900ae, negotiated
timeout = 20000
20790 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
20791 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
backtype.storm.zookeeper - Zookeeper state update: :connected:none
20792 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Session: 0x153c1a14d4900ae closed
20792 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.zookeeper.ClientCnxn - EventThread shut down
20793 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.utils.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs
[1000] the maxSleepTimeMs [30000] the maxRetries [5]
20793 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
20794 [storm.kafka.TestTopologyStaticHosts.main()] INFO
org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@39bb62e1
20795 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
20795 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
20796 [storm.kafka.TestTopologyStaticHosts.main(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/127.0.0.1:2181, sessionid = 0x153c1a14d4900af, negotiated
timeout = 20000
20796 [storm.kafka.TestTopologyStaticHosts.main()-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
20803 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.supervisor - Starting supervisor with id
af277158-e114-473c-bbcb-ce5c7ba5958a at host 192.168.1.11
20873 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.nimbus - Received topology submission for kafka-test with
conf {"topology.max.task.parallelism" nil, "topology.acker.executors" nil,
"topology.kryo.register" nil, "topology.kryo.decorators" (), "topology.name"
"kafka-test", "storm.id" "kafka-test-1-1459268133", "topology.debug" true}
20915 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.nimbus - Activating kafka-test: kafka-test-1-1459268133
21082 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.scheduler.EvenScheduler - Available slots:
(["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1028]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1029]
["b211e8c0-0b5b-4797-848c-abc7dc7bac6b" 1024]
["b211e8c0-0b5b-4797-848c-abc7dc7bac6b" 1025]
["b211e8c0-0b5b-4797-848c-abc7dc7bac6b" 1026])
21127 [storm.kafka.TestTopologyStaticHosts.main()] INFO
backtype.storm.daemon.nimbus - Setting new assignment for topology id
kafka-test-1-1459268133:
#backtype.storm.daemon.common.Assignment{:master-code-dir
"/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T//f11f892c-1a5c-4b06-8132-b2eca88dee75/nimbus/stormdist/kafka-test-1-1459268133",
:node->host {"af277158-e114-473c-bbcb-ce5c7ba5958a" "192.168.1.11"},
:executor->node+port {[2 2] ["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [3
3] ["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [4 4]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [5 5]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [6 6]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [7 7]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [8 8]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [9 9]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [10 10]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [11 11]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [12 12]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027], [1 1]
["af277158-e114-473c-bbcb-ce5c7ba5958a" 1027]}, :executor->start-time-secs {[2
2] 1459268134, [3 3] 1459268134, [4 4] 1459268134, [5 5] 1459268134, [6 6]
1459268134, [7 7] 1459268134, [8 8] 1459268134, [9 9] 1459268134, [10 10]
1459268134, [11 11] 1459268134, [12 12] 1459268134, [1 1] 1459268134}}
21836 [Thread-6] INFO backtype.storm.daemon.supervisor - Launching worker with
assignment #backtype.storm.daemon.supervisor.LocalAssignment{:storm-id
"kafka-test-1-1459268133", :executors ([2 2] [3 3] [4 4] [5 5] [6 6] [7 7] [8
8] [9 9] [10 10] [11 11] [12 12] [1 1])} for this supervisor
af277158-e114-473c-bbcb-ce5c7ba5958a on port 1027 with id
afbf5411-7a8e-4662-9fe7-6c8985501ab8
21839 [Thread-6] INFO backtype.storm.daemon.worker - Launching worker for
kafka-test-1-1459268133 on af277158-e114-473c-bbcb-ce5c7ba5958a:1027 with id
afbf5411-7a8e-4662-9fe7-6c8985501ab8 and conf {"dev.zookeeper.path"
"/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs"
10, "logviewer.childopts" "-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size"
1024, "storm.local.dir"
"/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T//d8d541cb-a98f-44c3-b09f-5a23a22daa67",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts"
true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs"
3600, "drpc.worker.threads" 64, "storm.meta.serialization.delegate"
"backtype.storm.serialization.DefaultSerializationDelegate",
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm",
"storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers"
("localhost"), "transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts"
"-Xmx768m", "supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 300,
"topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1027 1028 1029), "topology.environment" nil, "topology.debug" false,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port"
6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
"topology.disruptor.wait.timeout.millis" 1000, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1,
"storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport" "backtype.storm.messaging.netty.Context",
"logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080,
"nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144, "topology.classpath" nil}
21840 [Thread-6] INFO backtype.storm.utils.StormBoundedExponentialBackoffRetry
- The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
21841 [Thread-6] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
21841 [Thread-6] INFO org.apache.storm.zookeeper.ZooKeeper - Initiating client
connection, connectString=localhost:2181 sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@3adc3e01
26917 [Thread-6-SendThread(fe80:0:0:0:0:0:0:1%1:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181. Will not attempt to
authenticate using SASL (unknown error)
26918 [Thread-6-SendThread(fe80:0:0:0:0:0:0:1%1:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, initiating session
26919 [Thread-6-SendThread(fe80:0:0:0:0:0:0:1%1:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, sessionid =
0x153c1a14d4900b0, negotiated timeout = 20000
26919 [Thread-6-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
26919 [Thread-6-EventThread] INFO backtype.storm.zookeeper - Zookeeper state
update: :connected:none
26921 [Thread-6] INFO org.apache.storm.zookeeper.ZooKeeper - Session:
0x153c1a14d4900b0 closed
26921 [Thread-6-EventThread] INFO org.apache.storm.zookeeper.ClientCnxn -
EventThread shut down
26922 [Thread-6] INFO backtype.storm.utils.StormBoundedExponentialBackoffRetry
- The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
26922 [Thread-6] INFO
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl - Starting
26922 [Thread-6] INFO org.apache.storm.zookeeper.ZooKeeper - Initiating client
connection, connectString=localhost:2181/storm sessionTimeout=20000
watcher=org.apache.storm.curator.ConnectionState@495ed8b8
26923 [Thread-6-SendThread(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Opening socket connection to server
localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL
(unknown error)
26924 [Thread-6-SendThread(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Socket connection established to
localhost/0:0:0:0:0:0:0:1:2181, initiating session
26925 [Thread-6-SendThread(localhost:2181)] INFO
org.apache.storm.zookeeper.ClientCnxn - Session establishment complete on
server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x153c1a14d4900b1,
negotiated timeout = 20000
26925 [Thread-6-EventThread] INFO
org.apache.storm.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
26928 [Thread-6] INFO backtype.storm.daemon.worker - Reading Assignments.
26985 [Thread-6] INFO backtype.storm.daemon.worker - Launching receive-thread
for af277158-e114-473c-bbcb-ce5c7ba5958a:1027
26995 [Thread-7-worker-receiver-thread-0] INFO backtype.storm.messaging.loader
- Starting receive-thread: [stormId: kafka-test-1-1459268133, port: 1027,
thread-id: 0 ]
27403 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
print:[2 2]
27410 [Thread-6] INFO backtype.storm.daemon.task - Emitting: print __system
["startup"]
27410 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
print:[2 2]
27419 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor print:[2 2]
27427 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[3 3]
27429 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27429 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[3 3]
27436 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[3 3]
27444 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[4 4]
27446 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27446 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[4 4]
27448 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[4 4]
27455 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[5 5]
27456 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27456 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[5 5]
27457 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[5 5]
27463 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[6 6]
27465 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27465 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[6 6]
27468 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[6 6]
27473 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[7 7]
27512 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27512 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[7 7]
27513 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[7 7]
27518 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[8 8]
27520 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27520 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[8 8]
27522 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[8 8]
27528 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[9 9]
27530 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27530 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[9 9]
27534 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[9 9]
27539 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[10 10]
27541 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27541 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[10 10]
27544 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[10 10]
27552 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[11 11]
27553 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27553 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[11 11]
27555 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[11 11]
27566 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[12 12]
27567 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
27567 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
words:[12 12]
27569 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[12 12]
27585 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
__system:[-1 -1]
27588 [Thread-6] INFO backtype.storm.daemon.task - Emitting: __system __system
["startup"]
27588 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
__system:[-1 -1]
27590 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor __system:[-1 -1]
27619 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
__acker:[1 1]
27620 [Thread-6] INFO backtype.storm.daemon.task - Emitting: __acker __system
["startup"]
27621 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks
__acker:[1 1]
27623 [Thread-6] INFO backtype.storm.daemon.executor - Timeouts disabled for
executor __acker:[1 1]
27623 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor __acker:[1 1]
27634 [Thread-6] INFO backtype.storm.daemon.worker - Worker has topology
config {"storm.id" "kafka-test-1-1459268133", "dev.zookeeper.path"
"/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs"
10, "logviewer.childopts" "-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size"
1024, "storm.local.dir"
"/var/folders/t3/s0s2yw4j46qddz5bxq2cnp980000gq/T//d8d541cb-a98f-44c3-b09f-5a23a22daa67",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts"
true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs"
3600, "drpc.worker.threads" 64, "storm.meta.serialization.delegate"
"backtype.storm.serialization.DefaultSerializationDelegate",
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm",
"storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers"
("localhost"), "transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.kryo.decorators" (), "topology.name"
"kafka-test", "topology.transfer.buffer.size" 1024, "topology.worker.childopts"
nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 300,
"topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1027 1028 1029), "topology.environment" nil, "topology.debug" true,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.kryo.register" nil, "topology.message.timeout.secs" 30,
"task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts"
"-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1, "topology.disruptor.wait.timeout.millis"
1000, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1,
"storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport" "backtype.storm.messaging.netty.Context",
"logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080,
"nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144, "topology.classpath" nil}
27634 [Thread-6] INFO backtype.storm.daemon.worker - Worker
afbf5411-7a8e-4662-9fe7-6c8985501ab8 for storm kafka-test-1-1459268133 on
af277158-e114-473c-bbcb-ce5c7ba5958a:1027 has finished loading
27963 [refresh-active-timer] INFO backtype.storm.daemon.worker - All
connections are ready for worker af277158-e114-473c-bbcb-ce5c7ba5958a:1027 with
id afbf5411-7a8e-4662-9fe7-6c8985501ab8
27974 [Thread-15-words] INFO backtype.storm.daemon.executor - Opening spout
words:(5)
27986 [Thread-29-words] INFO backtype.storm.daemon.executor - Opening spout
words:(12)
27986 [Thread-17-words] INFO backtype.storm.daemon.executor - Opening spout
words:(6)
28004 [Thread-31-__system] INFO backtype.storm.daemon.executor - Preparing
bolt __system:(-1)
28020 [Thread-31-__system] INFO backtype.storm.daemon.executor - Prepared bolt
__system:(-1)
28031 [Thread-19-words] INFO backtype.storm.daemon.executor - Opening spout
words:(7)
28038 [Thread-33-__acker] INFO backtype.storm.daemon.executor - Preparing bolt
__acker:(1)
28038 [Thread-9-print] INFO backtype.storm.daemon.executor - Preparing bolt
print:(2)
28039 [Thread-9-print] INFO backtype.storm.daemon.executor - Prepared bolt
print:(2)
28039 [Thread-21-words] INFO backtype.storm.daemon.executor - Opening spout
words:(8)
28041 [Thread-33-__acker] INFO backtype.storm.daemon.executor - Prepared bolt
__acker:(1)
28049 [Thread-23-words] INFO backtype.storm.daemon.executor - Opening spout
words:(9)
28055 [Thread-11-words] INFO backtype.storm.daemon.executor - Opening spout
words:(3)
28056 [Thread-25-words] INFO backtype.storm.daemon.executor - Opening spout
words:(10)
28072 [Thread-27-words] INFO backtype.storm.daemon.executor - Opening spout
words:(11)
28072 [Thread-13-words] INFO backtype.storm.daemon.executor - Opening spout
words:(4)
28121 [Thread-11-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28121 [Thread-25-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28122 [Thread-29-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28122 [Thread-23-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28122 [Thread-13-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28122 [Thread-19-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28122 [Thread-21-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28122 [Thread-27-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28125 [Thread-17-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28125 [Thread-15-words] INFO
org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
28167 [Thread-13-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28167 [Thread-11-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28168 [Thread-13-words] WARN storm.kafka.KafkaUtils - Task [2/10] no
partitions assigned
28168 [Thread-29-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28195 [Thread-29-words] WARN storm.kafka.KafkaUtils - Task [10/10] no
partitions assigned
28167 [Thread-27-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28195 [Thread-27-words] WARN storm.kafka.KafkaUtils - Task [9/10] no
partitions assigned
28195 [Thread-29-words] INFO backtype.storm.daemon.executor - Opened spout
words:(12)
28196 [Thread-27-words] INFO backtype.storm.daemon.executor - Opened spout
words:(11)
28183 [Thread-21-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
28183 [Thread-15-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
28182 [Thread-17-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
28182 [Thread-23-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
28182 [Thread-27-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
28170 [Thread-13-words] INFO backtype.storm.daemon.executor - Opened spout
words:(4)
28168 [Thread-19-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28197 [Thread-19-words] WARN storm.kafka.KafkaUtils - Task [5/10] no
partitions assigned
28168 [Thread-21-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28198 [Thread-21-words] WARN storm.kafka.KafkaUtils - Task [6/10] no
partitions assigned
28198 [Thread-19-words] INFO backtype.storm.daemon.executor - Opened spout
words:(7)
28168 [Thread-25-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28198 [Thread-25-words] WARN storm.kafka.KafkaUtils - Task [8/10] no
partitions assigned
28168 [Thread-23-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28199 [Thread-23-words] WARN storm.kafka.KafkaUtils - Task [7/10] no
partitions assigned
28168 [Thread-15-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28199 [Thread-15-words] WARN storm.kafka.KafkaUtils - Task [3/10] no
partitions assigned
28199 [Thread-23-words] INFO backtype.storm.daemon.executor - Opened spout
words:(9)
28199 [Thread-15-words] INFO backtype.storm.daemon.executor - Opened spout
words:(5)
28199 [Thread-29-words] INFO backtype.storm.daemon.executor - Activating spout
words:(12)
28199 [Thread-15-words] INFO backtype.storm.daemon.executor - Activating spout
words:(5)
28168 [Thread-11-words] INFO storm.kafka.KafkaUtils - Task [1/10] assigned
[Partition{host=127.0.0.1:9092, partition=0}]
28168 [Thread-17-words] WARN storm.kafka.KafkaUtils - there are more tasks
than partitions (tasks: 10; partitions: 1), some tasks will be idle
28200 [Thread-17-words] WARN storm.kafka.KafkaUtils - Task [4/10] no
partitions assigned
28199 [Thread-19-words] INFO backtype.storm.daemon.executor - Activating spout
words:(7)
28199 [Thread-27-words] INFO backtype.storm.daemon.executor - Activating spout
words:(11)
28200 [Thread-17-words] INFO backtype.storm.daemon.executor - Opened spout
words:(6)
28199 [Thread-23-words] INFO backtype.storm.daemon.executor - Activating spout
words:(9)
28199 [Thread-13-words] INFO backtype.storm.daemon.executor - Activating spout
words:(4)
28199 [Thread-25-words] INFO backtype.storm.daemon.executor - Opened spout
words:(10)
28198 [Thread-21-words] INFO backtype.storm.daemon.executor - Opened spout
words:(8)
28200 [Thread-17-words] INFO backtype.storm.daemon.executor - Activating spout
words:(6)
28200 [Thread-21-words] INFO backtype.storm.daemon.executor - Activating spout
words:(8)
28200 [Thread-25-words] INFO backtype.storm.daemon.executor - Activating spout
words:(10)
33237 [Thread-13-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
33237 [Thread-11-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
33238 [Thread-29-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
33238 [Thread-25-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
33238 [Thread-19-words-EventThread] INFO
org.apache.curator.framework.state.ConnectionStateManager - State change:
CONNECTED
33252 [Thread-11-words] INFO storm.kafka.PartitionManager - Read partition
information from: /twitter/twitter-topic-id/partition_0 --> null
33420 [Thread-11-words] INFO storm.kafka.PartitionManager - No partition
information found, using configuration to determine offset
33420 [Thread-11-words] INFO storm.kafka.PartitionManager - Starting Kafka
127.0.0.1:0 from offset 185
33422 [Thread-11-words] INFO backtype.storm.daemon.executor - Opened spout
words:(3)
33422 [Thread-11-words] INFO backtype.storm.daemon.executor - Activating spout
words:(3)
57443 [Thread-11-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57452 [Thread-13-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57460 [Thread-15-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57471 [Thread-17-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57518 [Thread-19-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57524 [Thread-21-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57538 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57545 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57559 [Thread-27-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
57572 [Thread-29-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87448 [Thread-11-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87452 [Thread-13-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87460 [Thread-15-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87474 [Thread-17-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87522 [Thread-19-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87525 [Thread-21-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87540 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87546 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87561 [Thread-27-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
87575 [Thread-29-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
88027 [Thread-31-__system] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88031 [Thread-31-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@fa43cfa> [#<DataPoint
[__ack-count = {}]> #<DataPoint [memory/heap = {unusedBytes=208601008,
usedBytes=148963408, maxBytes=1908932608, initBytes=134217728,
virtualFreeBytes=1759969200, committedBytes=357564416}]> #<DataPoint [__receive
= {write_pos=1, read_pos=0, capacity=1024, population=1}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__execute-latency = {}]> #<DataPoint
[newWorkerEvent = 1]> #<DataPoint [__emit-count = {}]> #<DataPoint
[__execute-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [memory/nonHeap =
{unusedBytes=444112, usedBytes=60832048, maxBytes=136314880,
initBytes=24576000, virtualFreeBytes=75482832, committedBytes=61276160}]>
#<DataPoint [uptimeSecs = 97.933]> #<DataPoint [__transfer = {write_pos=0,
read_pos=0, capacity=1024, population=0}]> #<DataPoint [startTimeSecs =
1.459268103253E9]> #<DataPoint [__process-latency = {}]> #<DataPoint
[__transfer-count = {}]>]]
88043 [Thread-9-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88043 [Thread-33-__acker] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88043 [Thread-9-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@271bebf8> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=1,
read_pos=0, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency = {}]>
#<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]> #<DataPoint
[__execute-count = {}]>]]
88043 [Thread-33-__acker] INFO backtype.storm.daemon.task - Emitting: __acker
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@1a20fad7> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=1,
read_pos=0, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency = {}]>
#<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]> #<DataPoint
[__execute-count = {}]>]]
88202 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88202 [Thread-27-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88203 [Thread-19-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88202 [Thread-17-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88203 [Thread-19-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@29deebf3> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88203 [Thread-17-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@6a54b3fe> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88202 [Thread-13-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88204 [Thread-27-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@5ef51f3b> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88202 [Thread-21-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88204 [Thread-13-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@2ed58b31> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88203 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@587bd507> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88203 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88204 [Thread-21-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@45a555fc> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88202 [Thread-15-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88202 [Thread-29-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
88205 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@69b0371c> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88205 [Thread-15-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@2ff24579> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
88205 [Thread-29-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@4aaf2386> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
93428 [Thread-11-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
93431 [Thread-11-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@4c609700> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {Partition{host=127.0.0.1:9092,
partition=0}/fetchAPICallCount=32788, Partition{host=127.0.0.1:9092,
partition=0}/fetchAPILatencyMax=83, Partition{host=127.0.0.1:9092,
partition=0}/fetchAPILatencyMean=0.14252165426375504,
Partition{host=127.0.0.1:9092, partition=0}/fetchAPIMessageCount=0}]>
#<DataPoint [kafkaOffset = {partition_0/latestEmittedOffset=185,
totalLatestTimeOffset=185, totalSpoutLag=0, totalLatestEmittedOffset=185,
totalEarliestTimeOffset=131, partition_0/earliestTimeOffset=131,
partition_0/spoutLag=0, partition_0/latestTimeOffset=185}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
117453 [Thread-11-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117453 [Thread-13-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117461 [Thread-15-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117476 [Thread-17-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117526 [Thread-19-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117526 [Thread-21-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117545 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117546 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117564 [Thread-27-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
117579 [Thread-29-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147458 [Thread-11-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147459 [Thread-13-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147462 [Thread-15-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147480 [Thread-17-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147531 [Thread-21-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147531 [Thread-19-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147549 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147549 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147569 [Thread-27-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
147580 [Thread-29-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
148029 [Thread-31-__system] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148030 [Thread-31-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@7dcf27f3> [#<DataPoint
[__ack-count = {}]> #<DataPoint [GC/PSScavenge = {count=1, timeMs=18}]>
#<DataPoint [memory/heap = {unusedBytes=123031304, usedBytes=211464440,
maxBytes=1908932608, initBytes=134217728, virtualFreeBytes=1697468168,
committedBytes=334495744}]> #<DataPoint [__receive = {write_pos=2, read_pos=1,
capacity=1024, population=1}]> #<DataPoint [GC/PSMarkSweep = {count=0,
timeMs=0}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [newWorkerEvent = 0]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1,
read_pos=-1, capacity=1024, population=0}]> #<DataPoint [memory/nonHeap =
{unusedBytes=431000, usedBytes=61172840, maxBytes=136314880,
initBytes=24576000, virtualFreeBytes=75142040, committedBytes=61603840}]>
#<DataPoint [uptimeSecs = 157.932]> #<DataPoint [__transfer = {write_pos=0,
read_pos=0, capacity=1024, population=0}]> #<DataPoint [startTimeSecs =
1.459268103253E9]> #<DataPoint [__process-latency = {}]> #<DataPoint
[__transfer-count = {}]>]]
148049 [Thread-9-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148053 [Thread-33-__acker] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148055 [Thread-9-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@54d98329> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=2,
read_pos=1, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency = {}]>
#<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]> #<DataPoint
[__execute-count = {}]>]]
148057 [Thread-33-__acker] INFO backtype.storm.daemon.task - Emitting: __acker
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@46167717> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=2,
read_pos=1, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency = {}]>
#<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]> #<DataPoint
[__execute-count = {}]>]]
148207 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148208 [Thread-17-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148208 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@7bd75c8f> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148208 [Thread-19-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148210 [Thread-13-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148208 [Thread-29-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148210 [Thread-15-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148217 [Thread-15-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@2d387531> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148217 [Thread-13-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@2d387531> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148210 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148210 [Thread-21-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148218 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@5595fb28> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148210 [Thread-27-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
148210 [Thread-19-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@408ab946> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148210 [Thread-17-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@77fdefcd> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148219 [Thread-21-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@699c2afd> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148217 [Thread-29-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@2d387531> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
148238 [Thread-27-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@357e2b6b> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {}]> #<DataPoint [kafkaOffset =
{totalLatestTimeOffset=0, totalSpoutLag=0, totalLatestEmittedOffset=0,
totalEarliestTimeOffset=0}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
[__fail-count = {}]> #<DataPoint [__emit-count = {}]>]]
153434 [Thread-11-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
153436 [Thread-11-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@4a824b78> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaPartition = {Partition{host=127.0.0.1:9092,
partition=0}/fetchAPICallCount=36213, Partition{host=127.0.0.1:9092,
partition=0}/fetchAPILatencyMax=23, Partition{host=127.0.0.1:9092,
partition=0}/fetchAPILatencyMean=0.06392731891862038,
Partition{host=127.0.0.1:9092, partition=0}/fetchAPIMessageCount=0}]>
#<DataPoint [kafkaOffset = {partition_0/latestEmittedOffset=185,
totalLatestTimeOffset=185, totalSpoutLag=0, totalLatestEmittedOffset=185,
totalEarliestTimeOffset=131, partition_0/earliestTimeOffset=131,
partition_0/spoutLag=0, partition_0/latestTimeOffset=185}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
