Integrating OpaqueTridentKafkaSpout with Kafka & Java

Hi, Is there any good example for integrating OpaqueTridentKafkaSpout with Kafka ? I am using Storm 1.2.2 and Kafka 1.0 . Further I am using Tumbling window to batch every 100 messages and process them at once rather than working on each message from kafka as my tuple. I have posted my question

Re: How to use KafkaSpout to consume kafka cluster secured with kerberos

Yes, I faced the same issue with the latest storm Kafka client. Stig's patch helps to solve this issue. On Fri, 1 Jun 2018, 20:42 Stig Rohde Døssing, wrote: > Pretty sure it's this issue > https://issues.apache.org/jira/browse/STORM-3046. There's a link to a > patched 1.x branch you can use

Monitoring Apache Storm Trident Topology for Kafka Consumer Lags

. I realised that I cannot be using storm-kafka-monitor since offsets for Trident topologies are maintained in Storm Zookeeper rather than Kafka Zookeeper. Is there anyway to solve this? Thanks, Shaik Asifullah

Re: Monitoring Apache Storm Trident Topology for Kafka Consumer Lags

t you can visualize and > alert. > > Joshua > On Tue, Aug 21, 2018 at 3:18 AM Shaik Asifullah > wrote: > >> Hi, >> I am using Apache Storm Trident (1.2.1) for micro-batching. The main idea >> of my topology is to modify messages from Kafka a little and push it

Integrating Storm Trident and Kafka

Tick: Not enough data to calculate spout lag. 2018-04-09 14:17:34.129 o.a.s.k.KafkaUtils Thread-21-spout-spout-executor[88 88] [INFO] Metrics Tick: Not enough data to calculate spout lag. And in Storm UI, I can see acks for the messages above. Any suggestion to ignore metric Ticks ? Thanks, Shaik Asifullah

Re: Tuning Apache Storm Trident topology

igned Memory" to my Trident Topology ? I have posted my question here very long back https://stackoverflow.com/questions/50443737/increasing-assigned-memory-for-a-topology-in-storm . Please help me in resolving this issue. Thank you, Shaik Asifullah On Tue, Jul 31, 2018 at 1:57 AM Stig Rohde Døss

Re: storm-kafka-client Repeated consumption last message

Hi, Setting max spout pending value very high causes this. Try setting with low value say 1. My topology works good with 10 as max spout pending. More info here http://storm.apache.org/releases/1.1.2/FAQ.html Thanks, Shaik Asifullah On Tue, 20 Nov 2018, 07:23 731635762 Hi All, I use storm

Serialisation Issues Storm 1.2.1

) at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:748) Thanks, Shaik Asifullah

Re: Serialisation Issues Storm 1.2.1

Hi, Thanks Stephen for your response. Yes I doubled the previous max heap value but that didn't help me. Best Regards, Shaik Asifullah On Tue, Jun 11, 2019 at 1:35 PM Stephen Powis wrote: > Seems like you've exceeded your JVM's max heap settings. Maybe review and > increase the