Custom Grouping using flux

2018-04-04 Thread Rajeev
I am using a custom grouping that implements CustomStreamGrouping interface. Below is my chooseTasks method implementation: @Override public List< Integer > chooseTasks( int taskId, List< Object > values ) { List< Integer > boltIds = new ArrayList(); if( values.size() > 0 ) { long valueToGroup =

Re: sub millisecond latency with storm

2018-04-04 Thread Roshan Naik
Your latency observation is in line with  https://hortonworks.com/blog/microbenchmarking-storm-1-0-performance/  You might be able to achieve that kind of latency with the upcoming Storm 2.0. Since it is not released, you could try building the bits from the master branch. Some of the settings

Re: Storm throughput

2018-04-04 Thread Alessio Pagliari
> Something is definitely broken in your run or in your measurement method…. The problem doesn’t lie in my measurement method, I double checked trying as you said. Thank you for sharing the topo used, with that I was able to understand where I was failing. Basing my topo on other sample benchmar

sub millisecond latency with storm

2018-04-04 Thread Wijekoon, Manusha
In my topology I see around 1 - 2 ms latency when transferring tuples from spouts to bolts or from bolts to bolts. I am calculating latency using nanosecond timestamps because the whole topology runs inside a single worker. Topology is run in a cluster which runs in a production capable hardware