Re: Native Streaming Window API

Hi, You can look at the example code at https://raw.githubusercontent.com/apache/storm/master/examples/storm-starter/src/jvm/org/apache/storm/starter/SlidingWindowTopology.java and for trident at

[ANNOUNCE] Apache Storm 0.10.2 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.10.2. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project

batch processing in storm

Hi Is storm a right fit for batch processing - say every 10 min data from kafka source. does we need to right separate code for storm stream vs batch application ? Thanks

Re: Re: How will storm replay the tuple tree?

Very nice discussion ! I have also been wanting to see a feature something similar to Ravi's comment above: "*There is one thing i am looking forward from Storm is to inform Spout about what kind of failure it was*. i.e. if it was ConnectionTimeout or ReadTimeout etc, that means if i retry it

Kafka spout not working

Hi, I am not able to submit a topology that reads messages from kafka. Below is the error trace Storm version - 1.0.2 Kafka version - 0.10.0.0 17286 [SyncThread:0] ERROR o.a.s.s.o.a.z.s.NIOServerCnxn - Unexpected Exception: java.nio.channels.CancelledKeyException at

Re: storm-kafka-client 1.1.0 tag - Multiple Spouts Eventually Freeze

Zach, We have been seeing a similar issue with storm kafka since upgrade and posted about it earlier this week. We produced tests and opened pull request for feedback on what we saw. https://github.com/apache/storm/pull/1679 Regards James On Sep 12, 2016 12:03 PM, "Zach Schoenberger"

Re: Fwd: Serialize error when storm supervisor starting executor

Hi, Ambud, Thanks for your reply. I'm not using Maven to build my project. I tried excluding the conflict classes when packaging the jar, but the worker in supervisor then complains about NoClassDefException, though the class exists in one of the jars in the classpath of the worker. On Wed, Sep

Re: storm-kafka-client 1.1.0 tag - Multiple Spouts Eventually Freeze

I have seen that behavior only when running in local mode of storm and there is no data flowing in. This sounds like it might have something to do with zookeeper as in your offsets in zookeeper are either not updated or the watches are not being triggered for the spout to consume. Try using the

Re: Fwd: Serialize error when storm supervisor starting executor

Can you post the snippet of your pom.xml file especially around where storm-core is imported? I suspect you are not excluding dependencies explicitly if there is a conflict in maven . What is serialized is your bolt instance so you need either have serialization objects or mark them transient

Re: Join stream with historical data

Yes you can build something for data enrichment as long as your use some sort of LRU cache on the bolt that is fairly sizable and your event volume is reasonable to make sure there won't be a bottleneck in the topology. On Sep 13, 2016 10:43 AM, "Daniela S" wrote: > Dear

Re: [SURVEY] Which external modules (connector) do you use actively?

Storm JMS On Tue, Sep 13, 2016 at 5:30 PM, Kevin Conaway wrote: > We use the Kafka spout on Storm 0.10.0. > > We also make use of the Graphite metrics consumer library from Verisign. > > On Tue, Sep 13, 2016 at 4:57 AM, Jungtaek Lim wrote: > >> Hi

Re: How will storm replay the tuple tree?

Thank you guys for the discussion. What if I want exact-once processing for all nodes (bolts), even when failure happens, will Trident be the one? On Wed, Sep 14, 2016 at 3:49 PM, Ravi Sharma wrote: > Hi T.I. > Few things why Spout is responsible for replay rather then

Re: How will storm replay the tuple tree?

Hi T.I. Few things why Spout is responsible for replay rather then Various Bolts. 1. ack and fail messages carry only message ID, Usually your spouts generate messaged Id and knows what tuple/message is linked to it(via source i.e. jms etc). If ack or fail happens then Spout can do various