Thank you very much for your time. I will check it :) I did read that topology.max.spout.pending defaults to one and specifies the number of batches that can run over the topology simultaneously. Is that correct? What do you think is a good value to start testing? Just to have an idea.
Greetings! El 25/04/2014 18:43, "Danijel Schiavuzzi" <[email protected]> escribió: > From https://dl.dropboxusercontent.com/u/48250946/1.png it looks like the > two Kafka spout instances on host "pablo04" are having trouble consuming > from kafka -- their execute latencies are almost 30ms compared to the spout > instances on the other two hosts (4ms). > > You should check the Kafka broker on "pablo04" to determine why consuming > from it is so slow. > > Also, you should not leave "topology.max.spout.pending" unset. Be careful > though: when using Trident, it should be set to a number of _batches_, not > tuples (took me a while to figure this out :) > > > > On Fri, Apr 25, 2014 at 3:45 PM, Carlos Rodriguez <[email protected]>wrote: > >> Hi, >> I took new screenshots from Storm UI >> >> Topology Stats: https://dl.dropboxusercontent.com/u/48250946/2.png >> Spout3: https://dl.dropboxusercontent.com/u/48250946/1.png >> $mastercoord-bg3: https://dl.dropboxusercontent.com/u/48250946/4.png >> >> Topology Configuration: http://pastebin.com/uLsVa5Hn >> >> I hope these are useful, >> Thanks for your time! >> >> >> 2014-04-25 14:59 GMT+02:00 Danijel Schiavuzzi <[email protected]>: >> >> Could you provide the Storm UI stats for the "spout3" component? >>> >>> Also, it would be helpful if you could provide the topology >>> configuration. >>> >>> 4k messages/s is a very low throughput for Kafka, and should be easily >>> handled especially with a three-node broker. >>> >>> >>> >>> On Fri, Apr 25, 2014 at 1:43 PM, Carlos Rodriguez <[email protected]>wrote: >>> >>>> Hi, >>>> >>>> I'm using storm 0.9.1 and kafka 0.8. >>>> I've used trident to write a topology that get tuples from six >>>> partitions of a kafka topic. >>>> >>>> Every kafka message is about 2000 bytes on average, and that topic has >>>> like 4.000 messages per second. >>>> >>>> In the topology I use the kafka spout provided by wurstmeister at ( >>>> https://github.com/wurstmeister/storm-kafka-0.8-plus) with a >>>> parallelismHint of six, to match each executor with a partition. The >>>> version of the kafka spout is 0.5.0-SNAPSHOT (I updated it today to the >>>> last commit on the master branch). >>>> >>>> I use a cluster of 4 different machines to run the topology. Three of >>>> them run kafka (so they have 2 partitions for each machine) and >>>> supervisors, and the last of them runs the nimbus and others services that >>>> I need for my current setup, not related to storm nor kafka. >>>> >>>> I've configured the topology to use 3 works, and each supervisor only >>>> has one slot. >>>> >>>> The problem is that I have high latency problems on this configuration, >>>> and I'm pretty sure that it is related to kafka, because I've removed >>>> almost every line of code from my topology except the kafka spouts, and the >>>> problem persists. >>>> >>>> Let me show you the stats from storm-ui with a picture: >>>> https://dl.dropboxusercontent.com/u/48250946/stormScreenshot.png >>>> >>>> The last spout (spout3) is the kafka spout, and as you can see it has >>>> like 2 secs latency on each tuple. The $mastercoord-bg3 also have a lot of >>>> latency, and when I click on it, its "Output Stats" shows high latency on a >>>> stream called "$batch". >>>> >>>> I don't now if the problem is that the throughput is high for this >>>> configuration (4k msg/sec), or maybe that I have a low number of kafka >>>> partitions. >>>> >>>> I would appreciate any information about what is causing this and any >>>> tip about kafka & storm performance :) >>>> >>>> Thanks! >>>> >>> >>> >>> >>> -- >>> Danijel Schiavuzzi >>> >>> E: [email protected] >>> W: www.schiavuzzi.com >>> T: +385989035562 >>> Skype: danijels7 >>> >> >> > > > -- > Danijel Schiavuzzi > > E: [email protected] > W: www.schiavuzzi.com > T: +385989035562 > Skype: danijels7 >
