Re: Log Location for Storm UI

Please ignore my earlier mail .. Perhaps I just pasted the message without reading it fully ..Changing the *instance-1.c.apache-storm.internal* to my hostname did the trick :P ..I can now see those log messages .. On Sat, Oct 24, 2015 at 7:34 PM, Ankur Garg <ankurga...@gmail.com>

Re: Log Location for Storm UI

on. > > On Sat, Oct 24, 2015 at 5:28 PM, Ankur Garg <ankurga...@gmail.com> wrote: > >> Hi , >> >> I have set up a single node Storm Cluster to run my topologies . >> Unfortunately , due to reason unknown I am not seeing logs inside >> worker-*.log when

Re: binding to port 0.0.0.0/0.0.0.0:2181 !!

kill -9 Ex : sudo netstat -netulp | grep 2181 tcp0 0 0.0.0.0:*2181*0.0.0.0:* LISTEN 0 919692 17029/java then kill -9 17029 On Sat, Oct 24, 2015 at 8:04 PM, researcher cs wrote: > Thanks for replying , but the last

Re: Building fat jar for my test Topology to be deployed to remote Storm Cluster

SO , I got a workaround to solve this .. added exclude "META-INF/LICENSE" to my gradle file after checking the stack trace for the exception caused. If nyone has faced similar situation and has a better solution , please let me know. Thanks Ankur On Wed, Oct 14, 2015 at 5:59 PM,

Re: Does Storm work with Spring

declaring and initializing it inside each spout and bolts :( . Is there any reason why Spring Annotations dont work inside a Remote Cluster . Need help urgently here . Thanks Ankur On Sun, Oct 11, 2015 at 1:01 PM, Ankur Garg <ankurga...@gmail.com> wrote: > I think I don't need to Autowire be

Re: Does Storm work with Spring

PM, Ravi Sharma <ping2r...@gmail.com> wrote: > you may have to tell Spring that ur .yaml file is ur resource file. > > Ravi. > > On Mon, Oct 19, 2015 at 3:25 PM, Ankur Garg <ankurga...@gmail.com> wrote: > >> Hi Ravi , >> >> Need your help . So I

Re: Topology not working + Found multiple defaults.yaml resources when deploying to remote cluster

y > storm-core as a test dependency. > > I ended up going with the second option above and as part of the > packaging, I omit the storm-core jar from the fat jar. I'm not sure how > you would do that in gradle though. > > > > > On Wed, Oct 14, 2015 at 3:00 PM Ankur Gar

Shutting and Starting Storm Cluster

Hi , I have a single node storm cluster set up . Currently to start storm nimbus and supervisors daemons , I use storm nimbus and storm supervisor commands. To Stop it , currently I am doing kill -9 to kill those processes manually . Is there something I can use to restart cluster with one

Application Hangs inside Spout and Bolts

Hi , In my spout I am starting a spring boot application which basically initialises all the beans and classes . Whenever I start the topology in remote cluster , the moment this spring application runs inside the open method of my Spout , I see that the topology hangs and I see no errors and

Not able to see my application Logs in workerLogs

Hi , I have deployed my topology in remote cluster . Inside the open and prepare method for my spouts and bolts , I launch a separate application (Spring Application) which runs on a different port (just like zookeeper etc) and I get the instance of this application for use in my Spouts and

Re: Not able to see my application Logs in workerLogs

Any idea ppl . Even though application is running and my spouts and bolts are functioning , worker logs are stuck and nothing is getting printed there . On Tue, Oct 20, 2015 at 8:42 PM, Ankur Garg <ankurga...@gmail.com> wrote: > Hi , > > I have deployed my topology in

Re: Not able to see my application Logs in workerLogs

Thanks Javier for suggestion . Let me try this. Thanks Ankur On Wed, Oct 21, 2015 at 5:41 AM, Javier Gonzalez <jagon...@gmail.com> wrote: > Configure your cluster.xml to debug level for your own packages, set storm > debug to true, and retry. > On Oct 20, 2015 3:37 PM, "

Building fat jar for my test Topology to be deployed to remote Storm Cluster

Hi , I have set up a single node cluster and trying to deploy my sample topology to it . I believe to deploy my topology to cluster I have to submit the jar with all dependencies to the Cluster . For that I created a sample project and added a simple topology to it . While generating the fat

Re: Does Storm work with Spring

e > that. > > As John mentioned, its very important to mark all Spring beans and Context > as transient. > > Hope it helps. > > Ravi. > > > > > > On Sat, Oct 10, 2015 at 6:25 AM, Ankur Garg <ankurga...@gmail.com> wrote: > >> Hi Javier , >> >&

Re: Does Storm work with Spring

d some json where u can > define ur bolts and spouts and use that to build topology (u can use spring > but it's not much needed) > > In past I have done it using both json setting (without spring) and xml > setting (with spring) both works good > > Ravi > On 11 Oct 2015 06:38

Re: Does Storm work with Spring

s completely within a single JVM AFAIK. The local > cluster is useful for development, testing your topology, etc. The real > deployment has to go through nimbus, run on workers started by supervisors > on one or more nodes, etc. Kind of difficult to simulate all that on a > single box. > > On

Re: Multiple Spouts in Same topology or Topology per spout

them in two >> different topologies. That will allow me to independently deploy and scale >> the topology. But if the rest of logic is same I topology scaling and >> resource utilization will be better with one topology. >> >> I hope this helps.. >> >&g

Multiple Spouts in Same topology or Topology per spout

Hi , So I have a situation where I want to read messages from different queues hosted in a Rabbitmq Server . Now , there are three ways which I can think to leverage Apache Storm here :- 1) Use the same Spout (say Spout A) to read messages from different queues and based on the messages

Re: Multiple Spouts in Same topology or Topology per spout

> down one system because other is slower. > > > Last suggestion is to go with two spouts both will read from same > topic(not queue), so all messages will be delivered to both Spouts. One > Spout will send message to Mysql Bolt other will send to Mongo Bolt. > > &g

Re: Multiple Spouts in Same topology or Topology per spout

topology has enabled acknowledgment that means spout will always > receive > On 11 Oct 2015 18:15, "Ankur Garg" <ankurga...@gmail.com> wrote: > >> >> Thanks for the reply Abhishek and Ravi . >> >> One question though , going with One topology with mu

Re: Multiple Spouts in Same topology or Topology per spout

Kumar Gadalay <skgada...@gmail.com> wrote: > Check and insert > > On 10/12/15, Ankur Garg <ankurga...@gmail.com> wrote: > > But what if MongoDb bolt has some error , in that case I suppose the > entire > > tuple will be replayed from Spout meaning it will have to

Re: Multiple Spouts in Same topology or Topology per spout

ada...@gmail.com> wrote: > It is better to have 2 bolts - mysql bolt and mongodb bolt. > > Let mysql bolt forward the tuple to mongodb bolt, so in case of error > it won't emit. > > On 10/12/15, Ankur Garg <ankurga...@gmail.com> wrote: > > So I have a situation w

Processing failed message inside Storm

Hi , So I wanted to understand how ppl process messages which failed inside Storm . Do you store it in memory in some data structure (say queue) or use any other technique to store it . Please provide your inputs as to how you deal in production environment. Thanks Ankur

Exception causing Cluster to fail

Hi , I am testing out my topologies on my local Cluster . Unfortunately , whenever some exception occurs , my Cluster stops with error message like this : {"timeStamp":1444673373185,"host":"mum-1agrag-m.local","tid":"Thread-11-eventSaver","loglevel":"error","msg":"Async loop

Re: Exception Stack Trace for Local Cluster

t to do that > On Oct 6, 2015 2:07 PM, "Ankur Garg" <ankurga...@gmail.com> wrote: > >> Hi , >> >> I am running a local cluster on my dev machine . I see whenever something >> fails due to some exception in my code or library that I am using , I just >&

Re: Does Storm work with Spring

bolt and onOpen() method > in every Spout and make sure you don't use any autowire bean before this > call. > > > > > Ravi. > > > > > On Tue, Sep 29, 2015 at 2:22 PM, Ankur Garg <ankurga...@gmail.com> wrote: > > > Hi Ravi , > > > >

Setting up cluster on Google Cloud

Hi , I am trying to set up my cluster on Google Cloud and following below link : http://datadventures.ghost.io/2013/12/29/deploying-storm-on-gce/ But it looks like its 2 years old . In the meantime I try it out at my end , ny of u have idea how to deploy it on GCE and may be have some working

Exception Stack Trace for Local Cluster

Hi , I am running a local cluster on my dev machine . I see whenever something fails due to some exception in my code or library that I am using , I just get this message on console

Re: Does Storm work with Spring

le . But this works well in LocalCluster. My assumption is that it will work in a seperate Cluster too . Is my assumption correct ?? On Fri, Oct 9, 2015 at 9:04 PM, Javier Gonzalez <jagon...@gmail.com> wrote: > IIRC, only if everything you use in your spouts and bolts is serializabl

Evaluating Apache Storm

Hi , We in our organization are building a system to send real time notification (Pub-Sub) to Mobile Users(Android and IOS) . Broadly the use case is that mobile users subscribe to some topics and whenever there is/are events related to this topic , we need to send those users a Notification

Re: Evaluating Apache Storm

; So here you can use storm to process the events/notifications and then > push it from storm to different topics where the mobile users are > subscribed to. > > HTH > --Deepak > > On Mon, Aug 31, 2015 at 10:59 PM, Ankur Garg <ankurga...@gmail.com> wrote: > >> Hi , >&g

Deploying Apache Storm on Google Cloud

Hi , I wish to deploy storm cluster on Google Cloud . Browsing on the internet , I could find this => http://datadventures.ghost.io/2013/12/29/deploying-storm-on-gce/ Now , the above was written in 2013 . I read somewhere that Storm no longer needs Zmq instead it is using netty . Can someone

Emitting Custom Object as Tuple from spout

Hi , I am new to apache Storm . To understand it I was looking at storm examples provided in the storm tutorial ( https://github.com/apache/storm/tree/master/examples/storm-starter) . In all the examples , we are emitting prmitive types (String,int etc) as Tuples . In my use case I am having

Significance of boolean direct Output Fields declare (boolean direct, Fields fields)

Hi , Looking at the OutputFieldsdeclare class , I see there is an overloaded method for declare with a boolean flag direct . If i use the declare method i.e declare(Fields fields) ,it sets this boolean flag as false . I am not sure how storm interprets this boolean field internally while

Spring AMQP Integration with Storm

Hi, I want to consume the messages in my Storm Spout from a rabbitMq Queue. Now , we are using Spring AMQP to send and receive messages from RabbitMq asynchronously. Spring AMQP provides mechanism(either creating a listener or using annotation @RabbitListner) to read message from the queue

Re: Does Storm work with Spring

y own class where i have injected SpringContext so > SpringContext.getContext() returns the actuall Spring Context > > > > > Ravi. > > > On Tue, Sep 29, 2015 at 1:03 PM, Ankur Garg <ankurga...@gmail.com> wrote: > > > Hi , > > > > I am building a

Re: Spring AMQP Integration with Storm

wheel is worth the time and effort. > > Stephen > > On Tue, Sep 29, 2015 at 1:39 AM, Ankur Garg <ankurga...@gmail.com> wrote: > >> Hi, >> >> I want to consume the messages in my Storm Spout from a rabbitMq Queue. >> >> Now , we are using Spring AMQP to s

Does Storm work with Spring

Hi , I am building a Storm topology with set of Spouts and Bolts and also using Spring for Dependency Injection . Unfortunately , none of my fields are getting autowired even though I have declared all my spouts and Bolts as @Components . However the place where I am declaring my topology ,

Emitting Tuple to different Bolts

Hi , I am trying of a scenario in which I have a Spout which reads a data from a Message Broker and emits the message as a tuple to a Bolt for some Processing. Bolt post processing converts it into seperate Messages and each sub- message has to be sent to different Brokers which can be hosted

Re: Sending Some Context when Failing a tuple.

back(which i created when processing > started from that spout), so not sure how some other info either some > object or boolean or integer or String can be passed from Bolts to Spout > after failing. > > Thanks > Ravi > > > > > > > > > > On Sun, Jan 3

Re: Sending Some Context when Failing a tuple.

Hi Ravi, May be a very naive answer here but still posting it : I am assuming that once u catch the failed Tuple inside the fail method of your spout all you need to decide is whether this tuple should be replayed or not . I am assuming the object you are sending between spout and bolts are

Re: Multiple output streams: all go through the same connection?

Hi John , I am not aware of the storm internal working so I am not sure about this . But since no one has answered this so far , I will try my hand . I believe Spouts and Bolts communicate through an intermediate queue (ZMQ) . Assuming this it should always be a single connection to ZMQ . But

Apache Storm integration with Spring

Hi All , Sometime back I was working with Apache Storm in one of our Projects . Basically , the need was to run a topology over a Storm Cluster which consumes data from Rabbit Mq and does some processing over it . Processing also involved ingesting the feed from Rabbit Mq to some relational

Re: Introduce lag into a topology spout to process the events with a delay.

am not aware of full context of the problem too . * *Hope this helps* *Ankur* On Wed, 22 Feb 2017 at 11:22 Sandeep Samudrala <sandys...@gmail.com> wrote: > Yes. I am reading both the streams from kafka as part of a topology. > > On Wed, Feb 22, 2017 at 12:39 AM, Ankur Garg <an

Re: Introduce lag into a topology spout to process the events with a delay.

orking with a hack to check for the current backlog with the message header in the kafka event to look for events to be processed with a delay. Although it works to some extent I am still not able to get it fully working. Please let me know If I can add more context. On Wed, Feb 22, 2017 at 3:23 PM, Anku

Re: Introduce lag into a topology spout to process the events with a delay.

Hi Sandeep , One question :- how are you reading Streams B and A . Are u reading from some messaging queue (Kafka , Rabbit Mq etc.) with some spout (as part of some topology) reading from them . Please confirm . Thanks Ankur On Tue, 21 Feb 2017 at 15:28 Sandeep Samudrala

Re: Storm-related interview on Linux.com

Thanks Julien for sharing this and all the best for the conference :). On Wed, 12 Oct 2016 at 22:24 Julien Nioche wrote: > Hi, > > FYI, Linux.com have just published an interview [1] I gave on StormCrawler > in advance of the

unsubscribe

On Tue, Mar 26, 2019, 09:22 Romeo Nocon wrote: > unsubscribe >