Re: Storm fault tolerance benchmark

lable" to handle > partial node or network failure correctly. > > Does that answer your question? > > -- > Andrew Montalenti | CTO, Parse.ly > > On Sat, Aug 13, 2016 at 11:02 AM, Dominik Safaric <dominiksafa...@gmail.com > <mailto:dominiksafa...@gmail.c

Storm 2.0.0 release date?

Hi, what is the expected Storm 2.0.0 release date? Thanks, Dominik

Storm 1.0.2 - LoggingMetricsConsumer not reporting every n seconds

Hi, I’ve implemented a topology, where each of tasks/bolts report onto various metrics, including but not limited to number of tuples emitted. However, I’ve observed that the worker.log.metric files (when joined together from different workers), do not contain log entries for every second as

Storm 1.0.2 - KafkaBolt throughput skews

Hi Everyone,In the past few days, I’ve been benchmarking Storm using a simple topology consisting of a KafkaSpout and KafkaBolt. For the benchmark, I’ve produced 100.000.000 messages into Kafka, where each message was measured in 100 bytes. The configuration of Kafka, Zookeeper and Storm was

Storm 1.0.2 - when does Storm schedule additional workers?

Hi everyone, I’ve been curious about the following - under what conditions does Storm schedule additional workers of a running topology and what is its relation with the supervisor.slot.ports configuration value(s)? Thanks in advance, Dominik

KafkaSpout failed - stream: default not found

Hi, I’ve defined the following KafkaSpout: BrokerHosts hosts = new ZkHosts("localhost:2181"); SpoutConfig spoutConfig = new SpoutConfig(hosts, "bytes", "", UUID.randomUUID().toString()); spoutConfig.scheme = new RawMultiScheme(); KafkaSpout kafkaSpout = new KafkaSpout(spoutConfig);

Storm 1.0.2 - KafkaSpout not updating the offset/retrying tuples

H, I’ve implemented a topology consisting of a spout, processing bolt and a sink bolt pushing data back to Kafka. By examining the logs, I’ve seen that for all of the 12 partitions the totalSpoutLag and totalLatestTimeOffset remain constant (i.e. approximately at 830K), although tuples are

SpoutConfig zkRoot argument causing KafkaSpout exception

Hi, I’ve set up a topology consisting of a Kafka spout. But unfortunately, I keep getting the exception Caused by: java.lang.IllegalArgumentException: Invalid path string "//bytes3b68b144-e13c-4de3-beed-405e3ca5ae20/partition_1" caused by empty node name specified @1. Zookeeper has the

Re: SpoutConfig zkRoot argument causing KafkaSpout exception

ets will be > stored. > > >> On Sep 17, 2016 12:20 AM, "Dominik Safaric" <dominiksafa...@gmail.com> wrote: >> Hi, >> >> I’ve set up a topology consisting of a Kafka spout. But unfortunately, I >> keep getting the exception Caused by: java

KafkaSpout fails on open()

Hi, For the past two days, I’ve been tying to implement a KafkaSpout within our topology. Here are some important information. All three services are running on the same instance. Kafka’s brokers use as by default the 9092 port, with advertised.listeners set to PLAINTEXT://localhost:9092.

Re: Storm 1.0.2 - KafkaSpout cannot find partition information

gt; On Sep 17, 2016 9:09 AM, "Dominik Safaric" <dominiksafa...@gmail.com> wrote: >> Hi, >> >> I’ve deployed a topology consisting of a KafkaSpout using Kafka 0.10.0.1 and >> Zookeeper 3.4.6. All of the services, including the Nimbus and Supervisor, >> ru

Storm 1.0.2 - KafkaSpout cannot find partition information

Hi, I’ve deployed a topology consisting of a KafkaSpout using Kafka 0.10.0.1 and Zookeeper 3.4.6. All of the services, including the Nimbus and Supervisor, run on the same instance. However, by examining the worker.log file, I’ve found that the KafkaSpout is unable to find partitions

What determines the topology.acker.executors parameter value?

Hi Everyone, I’ve been curious onto the following - what determines the value of the topology.acker.executors parameter of the Storm configuration? By default, it is equal to the number of workers. However, if for example having a single worker, consisting of for example a spout with

Having the Nimbus and a worker on the same node of the cluster

Hi, It is generally recommended for streaming engines, also including Storm to run a separate master node - in the case of Storm, the Nimbus. However, why should one in Storm run the Nimbus on a separate node? Performance wise, the Nimbus isn’t intense unlike of course workers. In terms of

Re: Storm benchmarks

Hi Walid, You may benchmark Storm’s performance in terms of throughput and end-to-end latency for example. In addition, the investigation could also include variances in the configurational settings, such as the parallelism, message size, intra-worker and inter-worker buffer size which some of

Re: Storm benchmarks

inik, > > Many thanks for details. Actually I am looking for a set typologies for my > test. > > Thank you again, > -- > Regards > Walid > > > From: Dominik Safaric <dominiksafa...@gmail.com> > To: user@storm.apache.org; Walid Aljoby <walid_alj...@yaho

Storm 1.1.0 KafkaBolt writing only to a single partition

Hi, I would appreciate if anyone could explain the reason behind the following behaviour. I’m running a topology on a Storm cluster consisting of a nimbus and two workers nodes. The topology is comprised of a KafkaSpout reading messages from a Kafka topic having 8 partitions, and a KafkaBolt