Re: Is it possible to add/update/delete bolts at run time?

2018-05-19 Thread Artem Ervits
If there's no need for high volume and CEP capability in your bolts, perhaps you'd want to augment the architecture with something like Apache NiFi. You can dynamically add new connectors, be it Kafka or some RDBMS source. On Fri, May 18, 2018, 1:58 PM Abdul Samad

Re: Unit Testing Spouts + Bolts

2017-05-03 Thread Artem Ervits
in my experience, I used some of the examples in the Storm built-in tests to mock bolts. In that case, it wasn't necessary to run topologies. Here's an example of

Re: Visualizing Storm Metrics

2017-04-19 Thread Artem Ervits
Please consider looking at Apache Ambari, ships with Grafana to visualize latencies, queues, etc https://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.0.0/bk_ambari-user-guide/content/grafana_storm_topology.html On Apr 19, 2017 4:57 PM, "Cody Lee" wrote: > Currently I use

Re: Centralized logging for storm

2017-03-31 Thread Artem Ervits
If you deploy Storm with Ambari, you can leverage Ambari Log Search, we aggregate and parse the logs and present it in a nice UI. https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/viewing_service_logs.html The other benefits are prebuilt Grafana dashboards,

Re: HdfsSpout

2016-02-09 Thread Artem Ervits
does this help? https://github.com/apache/storm/tree/master/external/storm-hdfs On Tue, Feb 9, 2016 at 1:44 AM, K Zharas <kgzha...@gmail.com> wrote: > That is all about "HdfsBolt", not "HdfsSpout" > > On Tue, Feb 9, 2016 at 7:09 AM, Artem Ervits <artemerv...

Re: HdfsSpout

2016-02-08 Thread Artem Ervits
Here is some info http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_storm-user-guide/content/writing-data-with-storm-hdfs-connector.html On Feb 8, 2016 7:53 AM, "K Zharas" wrote: > Hi. > > 1) How can I set "HdfsSpout" so that it will emit a tuple every X seconds? >

Re: Binding port from code

2016-01-10 Thread Artem Ervits
You can overwrite config properties in your topology. Config.put(). On Jan 10, 2016 8:48 AM, "researcher cs" wrote: > is there any help ? > > On Sun, Jan 10, 2016 at 3:28 AM, researcher cs > wrote: > >> What if I want to write configuration

Re: Storm UI - topology visualization info

2015-10-30 Thread Artem Ervits
er. I always assumed it was the > % of how many tuples acked vs how many tuples emitted, but really I have > no idea. > > On Thu, Oct 29, 2015 at 5:18 PM, Artem Ervits <artemerv...@gmail.com> > wrote: > >> >> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3

Re: Storm UI - topology visualization info

2015-10-29 Thread Artem Ervits
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_storm-user-guide/content/deploy-and-manage-apache-storm-topologies.html On Oct 27, 2015 12:42 PM, "Adam Meyerowitz (BLOOMBERG/ 731 LEX)" < ameyerow...@bloomberg.net> wrote: > Hi, can someone help me understand the information that's