Re: Resource Aware Scheduler Kills Nimbus

Hi Wuyang, This is a nice question, I didn’t find anything that computes it automatically, yet. By now you have to test your topology tuning the resource requirements until you don’t find pleasing values. Cheers, Alessio > On 10 Feb 2018, at 21:42, Wuyang Zhang wrote: > > Hi Jerry, > > I

Storm throughput

Hi everybody, I’m trying to do some preliminary tests with storm, to understand how far it can go. Now I’m focusing on trying to understand which is his maximum throughput in terms of tuples per second. I saw the benchmark done by the guys at Hortonworks (ref: https://it.hortonworks.com/blog/m

Re: Storm throughput

; >Jacob Johansen > >On Fri, Mar 30, 2018 at 9:18 AM, Alessio Pagliari > >wrote: > >> Hi everybody, >> >> I’m trying to do some preliminary tests with storm, to understand how >far >> it can go. Now I’m focusing on trying to understand which is his >ma

Re: Storm throughput

ple benchmark topologies that I found online, I enabled the option setDebug(true); printing an output message for each tuple was slowing me down, now I’m able to reach a spout emission rate of ~4.5M tuples per second. Thank you all for the support. -- Alessio Pagliari Scale Team, PhD Stud

Re: Running multiple topologies

, -- Alessio Pagliari Scale Team, PhD Student Université Côte d’Azur, CNRS, I3S > On 12 Apr 2018, at 20:51, Ethan Li wrote: > > Hi Daniel, > > I am not sure if I understand your questions correctly. But will the resource > aware scheduler help? > https://github.com/apa

Storm benchmarks: UI stat updates

where I can deepen this aspect. Thank you, -- Alessio Pagliari Scale/Signet Team, PhD Student Université Côte d’Azur, CNRS, I3S Website: http://www.i3s.unice.fr/~pagliari/ <http://www.i3s.unice.fr/~pagliari/>

New Metrics2 error in documentation

Hello, While trying the new Metrics2 in Storm 1.2.1, I noticed an error in the online documentation (http://storm.apache.org/releases/1.2.1/metrics_v2.html ): > > CSV Reporter (org.apache.storm.metrics2.reporters.CsvReporter): Reports >

Display ackers executors

Hello everybody, I cannot find a way to show in the UI or retrieve through REST APIs the placement of the acker executors. Is there a way to get this information? I remember that in some topologies in the past I was able to see them in the UI with the name ‘__acker’, but now I can’t understand

Re: Display ackers executors

st, > Ethan > >> On Jul 19, 2018, at 11:04 AM, Alessio Pagliari wrote: >> >> Hello everybody, >> >> I cannot find a way to show in the UI or retrieve through REST APIs the >> placement of the acker executors. Is there a way to get this information? I

Re: Display ackers executors

> <https://fubariteblue-ni.blue.ygrid.yahoo.com:4443/api/v1/topology/cb_reporting_cow_updates-612-1531172361/component/__acker?sys=true> > > > > >> On Jul 19, 2018, at 11:50 AM, Alessio Pagliari > <mailto:pagli...@i3s.unice.fr>> wrote: >> >> Thank you Ethan, I tota

Modifying tasks number during scheduling

Hi guys, I was playing a bit with a Custom Scheduler to understand how it works. I was wondering, is there a way to change the number of tasks for a certain component from the Scheduler class? Like for example if during the scheduling I'd like to increase or decrease the number of the spouts?

Re: When is the MAX_SPOUT_PENDING limit applied

Hi Jayant, The MAX_SPOUT_PENDING is applied in each of the spouts. Each spout take count of all the tuples it sends, increasing it for each emit and decreasing it when it receive the final ack message for a “pending tuple” (= a tuple that is sent but it’s still waiting an ack). If the number