Re: Metrics Instrumentation

2016-09-21 Thread Otis Gospodnetić
Hi Joaquin, Hi Cody :) If you need something that works out of the box, try SPM - https://sematext.com/spm/ Otis -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ On Wed, Sep 21, 2016 at 9:31 AM, Cody Lee

Re: Submitting Topology Questions (2 small questions)

2016-09-21 Thread Ambud Sharma
Just the nimbus address and port On Sep 21, 2016 6:50 PM, "Joaquin Menchaca" wrote: > What is the minimal storm.yaml configuration do I need for `storm jar ... > remote`? > > Is there a command line option or way to specify locally crafted > storm.yaml, e.g.

Submitting Topology Questions (2 small questions)

2016-09-21 Thread Joaquin Menchaca
What is the minimal storm.yaml configuration do I need for `storm jar ... remote`? Is there a command line option or way to specify locally crafted storm.yaml, e.g. /tmp/storm.yaml? -- 是故勝兵先勝而後求戰，敗兵先戰而後求勝。

Re: Cannot submit topology in local mode on Storm 1.0.1

2016-09-21 Thread Jungtaek Lim
Unfortunately without the minimal code which can reproduce this I'm hard to dig more. I followed the path based on stack trace, and it would be IllegalArgumentException, but it should have message which describes why it is thrown. Do you find more stack trace informations on that? - Jungtaek Lim

Re: Large number of very small streams

2016-09-21 Thread Ambud Sharma
Two solutions: 1. You can group users by some sort of classification and create topics based on that then for each user the consumer can check if it's interested in the topic and consumer or reject the messages. 2. If each user writes a lot of data then you can use the concept of key based

Large number of very small streams

2016-09-21 Thread Ivan Gozali
Hi everyone, I'm very new to Storm, and have read various documentation but haven't started using it. I have a use case where I could potentially have many users producing data points that are accumulated in one huge, single Kafka topic/Kinesis stream, and I was going to use Storm to "route"

Re: Metrics Instrumentation

2016-09-21 Thread Cody Lee
If you need some topology metrics and custom metrics, I’ve updated the older storm-metrics-statsd reporter for the 1.x version of storm: https://github.com/platinummonkey/storm-metrics-statsd and provided an example working with for example datadog here:

Re: Syncing multiple streams to compute final result from a bolt

2016-09-21 Thread Harsh Choudhary
It is real-time. I get streaming JSONs from Kafka. On Wed, Sep 21, 2016 at 4:15 AM, Ambud Sharma wrote: > Is this real-time or batch? > > If batch this is perfect for MapReduce or Spark. > > If real-time then you should use Spark or Storm Trident. > > On Sep 20, 2016