Hi all,
I want to enhance the performance of the running topology.
If i done all standard configuration and then if i want to improve the
performance then i have to
re-balance and schedule the topology of the basis of traffic flow on the
bolts.
As i new to storm. am I going in right way?
I
I am getting this error message in the Storm UI. Topology works fine on
localCluster.
java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 3 at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
at
Can someone tell me why people put Kafka in front of Storm? Can’t Storm ingest
messages without having Kafka in the middle?
B.
I get your reasoning at a high level. I should have specified that I wasn’t
sure what Kafka does. I don’t have a hard software engineering background. I
know that Kafka is “a message queuing” system, but I don’t really know what
that means.
(I can’t believe you wrote all that from your
If you are familiar with Weblogic or ActiveMQ, it is similar. Let's see if
I can explain, I am definitely not a subject matter expert on this.
Within Kafka you can create queues, ie a webclicks queue. Your web
servers can then send click events to this queue in Kafka. The web servers,
or agent
Also, since Kafka acts as a buffer, storm is not directly affected by the speed
of your data sources/producers.
-Original Message-
From: Justin Workman justinjwork...@gmail.com
Sent: 15-08-2014 07:12
To: user@storm.incubator.apache.org user@storm.incubator.apache.org
Subject: Re:
I suppose not directly. It depends on the lifetime of your Kafka queues
and on your latency requirements. You need to make sure you have enough
doctors or in storm language workers, in your storm cluster to process
your messages within your SLA.
For our case we, we have a 3 hour lifetime or ttl
I agree, not for the long run but for small bursts in data production rate, say
peak hours, Kafka can help in providing a somewhat consistent load on Storm
cluster.
-Original Message-
From: Justin Workman justinjwork...@gmail.com
Sent: 15-08-2014 07:53
To:
Absolutely!
Sent from my iPhone
On Aug 14, 2014, at 9:02 PM, anand nalya anand.na...@gmail.com wrote:
I agree, not for the long run but for small bursts in data production rate,
say peak hours, Kafka can help in providing a somewhat consistent load on
Storm cluster.
Kafka is also distributed in nature, which is not something easily achieved
by queuing brokers like ActiveMQ or JMS (1.0) in general. Kafka allows data
to be partitioned across many machines which can grow as necessary as your
data grows.
On Thu, Aug 14, 2014 at 11:20 PM, Justin Workman
I have a situation where i have seven bolts and one spout i want to
distribute the tuples according to the field ID.
For eg. if ID=21 I want the tuple to be processed by first bolt
ID=31 I want that tuple to be processed by second bolt so
on.
So is there a way to implement these.
11 matches
Mail list logo
