Sasikumar created STORM-293:
-------------------------------
Summary: Relatime Topologies Time frequency
Key: STORM-293
URL: https://issues.apache.org/jira/browse/STORM-293
Project: Apache Storm (Incubating)
Issue Type: Bug
Environment: CentOS, 1 Nimbus and 3 Supervisor
Reporter: Sasikumar
We are running realtime trident topologies that would listen to the Database
for new / updated records at every 5 minutes and then send to bolts for some
operations. We use Utils.Sleep(300000) of Storm to keep the spout sleep for 5
minutes.
TridentTopology topology = new TridentTopology();
topology.newStream("Migration",new
MigrationAuditor()).name("MigrationAuditorSpout").parallelismHint(1).shuffle()
.name("MigratorBolt").each(new
Fields("vcRelationsVO","vcendTimeStamp","vcskip"),new VCMigrator(),new
Fields("vcendtime","vccnt")).parallelismHint(2).shuffle()
.name("LastRunBoltVC").each(new Fields("vcendtime","vccnt"),new
LastRunCalculatorForVC(),new Fields("vcmlpaData")).parallelismHint(1).shuffle();
The issue that we face are given below:
1. The spout is getting down randomly at sometime and not runs continuously for
every 5 minutes.
2. The bolts (Trident Function) that receives the data from the Spout goes down
randomly and hence missing the continuity / data.
For example, when the spout picked up 5 events and goes to sleep, the
VCMigrator in this example process 2 events and goes down, there are no traces
in log. Sometimes none of the events are processed by the bolt and no traces
are found in log. We could not see any log related to this bolt after it goes
down. This happens for all the realtime topologies that we run and not only
with one.
This runs perfect in the version 0.8.2 and is creating problem in 0.9.0.1 and
0.9.1.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
