Yes running from command line gives me json with topic lag. What is secured setup?
On Wed, Feb 1, 2017 at 10:28 PM, Priyank Shah <[email protected]> wrote: > Hi Igor, > > > > Just to clarify, are you saying that running storm-kafka-monitor from > command line with the arguments you pasted returns the spout lags correctly > ? Also, can you confirm if you are running in a secured setup or not ? > > > > *From: *Igor Kuzmenko <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Wednesday, February 1, 2017 at 5:05 AM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Kafka monitor unable to get offset lag > > > > Found in logs with debug level: > > 2017-02-01 14:06:54.053 o.a.s.u.TopologySpoutLag [DEBUG] spout classname: > org.apache.storm.kafka.spout.KafkaSpout > > 2017-02-01 14:06:54.053 o.a.s.u.TopologySpoutLag [DEBUG] json > configuration: {config.bootstrap.servers=dn-06.ru:6667,dn-10.ru:6667, > config.topics=gtp,, config.groupid=test, topology.tasks=1} > > 2017-02-01 14:06:54.053 o.a.s.u.TopologySpoutLag [DEBUG] Command to run: > [/usr/hdp/2.5.0.0-1245/storm/bin/storm-kafka-monitor, -t, gtp,, -g, test, > -b, dn-06.ru:6667,dn-10.ru:6667] > > 2017-02-01 14:07:36.338 o.a.s.u.TopologySpoutLag [DEBUG] JSON parsing > failed, assuming message as error message: Unable to get offset lags for > kafka. Reason: org.apache.kafka.shaded.common.errors.TimeoutException: > Timeout expired while fetching topic metadata > > > > Running command /usr/hdp/2.5.0.0-1245/storm/bin/storm-kafka-monitor -t > gtp -g test -b dn-06.ru:6667,dn-10.ru:6667 returns correct json with > spout lags. There's no timeout command executing in a second. > > > > > > > > On Wed, Feb 1, 2017 at 12:25 PM, Spico Florin <[email protected]> > wrote: > > Hello! > > You can check how your topic is consumed and its health via the scripts: > > ./kafka-consumer-groups.sh --new-consumer --bootstrap-server > <your-broker>:6667 --list > > ./kafka-consumer-groups.sh --new-consumer --bootstrap-server > <your-broker>:6667 --describe --group <your-group> > > > > I hope it helps. > > Florin > > > > On Wed, Feb 1, 2017 at 11:01 AM, Igor Kuzmenko <[email protected]> wrote: > > Yes, topology process data and works fine. > > I couldn't find any exceptions in storm logs. access-web-ui.log contains > only these lines > > > > 2017-02-01 11:58:17.315 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: > 10.35.63.14 url: http://master001.s:8744/api/v1/cluster/summary principal: > > 2017-02-01 11:58:17.315 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: > 10.35.63.14 url: http://master001.s:8744/api/v1/nimbus/summary principal: > > 2017-02-01 11:58:25.346 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: > 10.35.63.14 url: http://master001.s:8744/api/v1/topology/summary > principal: > > 2017-02-01 11:58:25.346 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: > 10.35.63.14 url: http://master001.s:8744/api/v1/cluster/summary principal: > > 2017-02-01 11:58:25.346 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: > 10.35.63.14 url: http://master001.s:8744/api/v1/nimbus/summary principal: > > > > On Tue, Jan 31, 2017 at 5:19 PM, Priyank Shah <[email protected]> > wrote: > > Hi Igor, > > > > When you say topology is working fine do you mean you see data flowing? > Can you try to look up the logs for ui server and paste relevant lines here > if any? > > > > Priyank > > Sent from my iPhone > > > On Jan 31, 2017, at 4:34 AM, Igor Kuzmenko <[email protected]> wrote: > > I've launched topology with new kafka spout. Topology by it self working > fine, but looking at storm UI I see kafka-monitor exception: > > *Unable to get offset lags for kafka. Reason: > org.apache.kafka.shaded.common.errors.TimeoutException: Timeout expired > while fetching topic metadata* > > > > Maybe I forgot to configure something, but then how topology reads > messages? > > > > > > >
