why storm still use it's own zookeeper node not my zookeeper cluster?

hi,maillist: i am a storm newbie ,when i start storm nimbus and suervisor ,i check log and find the following info 2014-04-08 09:26:02 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT 2014-04-08 09:26:02 o.a.z.s.ZooKeeperServer

Fwd: Storm with Drools Complex Event Processing rules

Hi, I am using Storm with Drools Fusion for adding CEP rules to raise system alerts . I need to raise alerts for system if condition is matched for last 10 records or 10 minutes. Bolt is receiving one event in tuple from Spout. How can I make it work where bolt can read multiple events(lets say

Re: why storm still use it's own zookeeper node not my zookeeper cluster?

Where is the storm.yaml placed? Do you have a ~/.storm/storm.yaml with overrides? Niels On Apr 24, 2014 8:05 AM, ch huang justlo...@gmail.com wrote: hi,maillist: i am a storm newbie ,when i start storm nimbus and suervisor ,i check log and find the following info 2014-04-08

Storm dies when zookeeper followers are not available

We have a cluster deployment of Storm and Zookeeper. Where storm is running on 2 machines and zookeeper is running on 3 machines. Now if the zookeeper followers are not available, though the leader is available storm keeps throwing an exception that it is not able to connect to zookeeper; and

Re: Storm with Drools Complex Event Processing rules

Hi Nishi, I will point to you some resources that solve similar problems: 1. Book: Storm Real-time processing Cookbook http://amzn.to/OEaK5N has chapter Rule-based analysis of the log stream. This recipe integrates the JBoss Library and Drools into a bolt and allow to capture event based on rule.

RE: Logging levels

If you want to adjust on a per topology basis create a logback config and package in uber jar The config should be an included logback config file rather than stand alone, so root tag is included After that all storm cluster.xml files should be updated to include include optional=true

Re: storm starter ExclamationTopology

and the question is , what should I fix dear experts ? :) 2014-04-24 16:23 GMT+00:00 Bilal Al Fartakh alfartaj.bi...@gmail.com: ~/src/storm-0.8.1/bin/storm jar /root/src/storm-starter/target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.ExclamationTopology demo *I

Re: storm starter ExclamationTopology

In your storm cluster , you need to verify first, nimbus is running properly or not. Check nimbus.log in $STORM_HOME/logs directory for error logs. Also check nimbus.host parameter in ~/.storm/storm.yaml. On Thu, Apr 24, 2014 at 9:56 PM, Bilal Al Fartakh alfartaj.bi...@gmail.comwrote: and the

Re: storm starter ExclamationTopology

In your storm cluster , you need to verify first, nimbus is running properly or not. Check nimbus.log in $STORM_HOME/logs directory for error logs. Also check nimbus.host parameter in ~/.storm/storm.yaml. Yeah, that's what I was writing in my reply. I'll go ahead and add below: Start nimbus

time aggregated data and shared structures efficiency

I have already developed a Python script (not using storm) which transforms a stream of millions of prices history of different items (provided in 1 common csv) and output dedicate streams for each item with enriched data in real-time. This script computes and aggregates in real-time latest item

Re: Storm with Drools Complex Event Processing rules

Hi Eugene, Thanks for the links. These links are quite useful to go forward in this direction. As these books are paid if you have copy of these books available, can you please share on my mail ID? Thanks, Nishu On Thu, Apr 24, 2014 at 6:59 PM, Eugene edvor...@gmail.com wrote: Hi Nishi, I

Trident Stream Join with Self

Is it possible to join a trident stream with itself? My particular use case is that I want to take the cross product of all the incoming tuples for a batch and then only keep the joined tuples containing a known value. I believe the SQL for what I am trying to accomplish is: SELECT * FROM table

Re: Flush aggregated data every X seconds

Raphael, in your case it sounds like a TickSpout could be useful where you emit a tuple every n time slices and then sleep until needing to emit another. I'm not sure how that'd work in a Trident aggregator, however. I'm not sure if this is something Nathan or the community would approve of, but

Re: Flush aggregated data every X seconds

Thank you very much for your quick reply Corey, Unfortunately I don't believe the TickSpout exists within Trident yet. I have seen the threads discussing the implementation of a sliding window and I've read Michael Noll's

Re: Storm dies when zookeeper followers are not available

Once you lose the zookeeper quorum, I have seen that the workers keep throwing exceptions that they are not able to connect to the zookeeper. But I haven't seen them die because of this. I have even seen them recover once the quorum is restored. But if the worker process gets killed, they don't