[
https://issues.apache.org/jira/browse/STORM-2568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haewan Jeong updated STORM-2568:
--------------------------------
Description:
Hello
I've tried to use storm-kafka-monitor, and it works fine on command line If I
changed 'toollib/storm-kafka-monitor-*.jar' to
'toollib/storm-kafka-monitor-1.1.0.jar'.
{code}
{"my-kafka-topic-name":{"0":{"consumerCommittedOffset": 74804998,
"logHeadOffset": 74805483, "lag": 485},"1":{"consumerCommittedOffset":
74804998, "logHeadOffset": 74805485, "lag":
487},"2":{"consumerCommittedOffset": 74804995, "logHeadOffset": 74805485,
"lag": 490},"3":{"consumerCommittedOffset": 74805001, "logHeadOffset":
74805488, "lag": 487},"4":{"consumerCommittedOffset": 74805011,
"logHeadOffset": 74805484, "lag": 473},"5":{"consumerCommittedOffset":
74805009, "logHeadOffset": 74805485, "lag":
476},"6":{"consumerCommittedOffset": 74805008, "logHeadOffset": 74805483,
"lag": 475},"7":{"consumerCommittedOffset": 74805010, "logHeadOffset":
74805484, "lag": 474},"8":{"consumerCommittedOffset": 73641446,
"logHeadOffset": 74805488, "lag": 1164042},"9":{"consumerCommittedOffset":
73641448, "logHeadOffset": 74805489, "lag":
1164041},"10":{"consumerCommittedOffset": 73641443, "logHeadOffset": 74805483,
"lag": 1164040},"11":{"consumerCommittedOffset": 73641445, "logHeadOffset":
74805487, "lag": 1164042},"12":{"consumerCommittedOffset": 74805003,
"logHeadOffset": 74805486, "lag": 483},"13":{"consumerCommittedOffset":
74804999, "logHeadOffset": 74805482, "lag":
483},"14":{"consumerCommittedOffset": 74805002, "logHeadOffset": 74805483,
"lag": 481},"15":{"consumerCommittedOffset": 74805002, "logHeadOffset":
74805484, "lag": 482},"16":{"consumerCommittedOffset": 74804994,
"logHeadOffset": 74805482, "lag": 488},"17":{"consumerCommittedOffset":
74805002, "logHeadOffset": 74805489, "lag":
487},"18":{"consumerCommittedOffset": 74805003, "logHeadOffset": 74805488,
"lag": 485},"19":{"consumerCommittedOffset": 74805003, "logHeadOffset":
74805489, "lag": 486}}}
{code}
but it gives empty result when I call below api.
{code}
/api/v1/topology/:id/lag
...
{
"MySpoutName": {
"spoutLagResult": {},
"spoutId": "MySpoutName",
"spoutType": "KAFKA"
}
}
{code}
-I think that needs to fix "groupid" to "group.id" in TopologySpoutLag.java I
debug it, but groupid is right.-
the reason was topics has square brackets in command.
{code}
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] json
configuration: {config.security.protocol=null,
config.bootstrap.servers=kafka.xxx.com:9092, config.topics=[my-kafka-topic],
config.groupid=my-storm-kafka-spout-groupid, topology.tasks=5}
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
/my/program/storm/bin/storm-kafka-monitor
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -t
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
[my-kafka-topic]
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -g
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
my-storm-kafka-spout-groupid
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -b
2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
kafka.xxx.com:9092
{code}
the square brackets automatically added because of this
{code}
package org.apache.storm.kafka.spout;
public class NamedSubscription extends Subscription {
...
@Override
public String getTopicsString() {
return String.valueOf(topics);
}
{code}
topics is Collections. so String.valueOf returns value with square brackets.
I fixed the code that remove square brackets in TopologySpoutLag.java. but I
think that fixing 'getTopicsString of NamedSubscription.java in
org.apache.storm.kafka.spout' is might be better.
was:
Hello
I've tried to use storm-kafka-monitor, and it works fine on command line If I
changed 'toollib/storm-kafka-monitor-*.jar' to
'toollib/storm-kafka-monitor-1.1.0.jar'.
{code}
{"my-kafka-topic-name":{"0":{"consumerCommittedOffset": 74804998,
"logHeadOffset": 74805483, "lag": 485},"1":{"consumerCommittedOffset":
74804998, "logHeadOffset": 74805485, "lag":
487},"2":{"consumerCommittedOffset": 74804995, "logHeadOffset": 74805485,
"lag": 490},"3":{"consumerCommittedOffset": 74805001, "logHeadOffset":
74805488, "lag": 487},"4":{"consumerCommittedOffset": 74805011,
"logHeadOffset": 74805484, "lag": 473},"5":{"consumerCommittedOffset":
74805009, "logHeadOffset": 74805485, "lag":
476},"6":{"consumerCommittedOffset": 74805008, "logHeadOffset": 74805483,
"lag": 475},"7":{"consumerCommittedOffset": 74805010, "logHeadOffset":
74805484, "lag": 474},"8":{"consumerCommittedOffset": 73641446,
"logHeadOffset": 74805488, "lag": 1164042},"9":{"consumerCommittedOffset":
73641448, "logHeadOffset": 74805489, "lag":
1164041},"10":{"consumerCommittedOffset": 73641443, "logHeadOffset": 74805483,
"lag": 1164040},"11":{"consumerCommittedOffset": 73641445, "logHeadOffset":
74805487, "lag": 1164042},"12":{"consumerCommittedOffset": 74805003,
"logHeadOffset": 74805486, "lag": 483},"13":{"consumerCommittedOffset":
74804999, "logHeadOffset": 74805482, "lag":
483},"14":{"consumerCommittedOffset": 74805002, "logHeadOffset": 74805483,
"lag": 481},"15":{"consumerCommittedOffset": 74805002, "logHeadOffset":
74805484, "lag": 482},"16":{"consumerCommittedOffset": 74804994,
"logHeadOffset": 74805482, "lag": 488},"17":{"consumerCommittedOffset":
74805002, "logHeadOffset": 74805489, "lag":
487},"18":{"consumerCommittedOffset": 74805003, "logHeadOffset": 74805488,
"lag": 485},"19":{"consumerCommittedOffset": 74805003, "logHeadOffset":
74805489, "lag": 486}}}
{code}
but it gives empty result when I call below api.
{code}
/api/v1/topology/:id/lag
...
{
"MySpoutName": {
"spoutLagResult": {},
"spoutId": "MySpoutName",
"spoutType": "KAFKA"
}
}
{code}
-I think that needs to fix "groupid" to "group.id" in TopologySpoutLag.java I
debug it, but groupid is right.-
> 'api/vi/topology/:id/lag' returns empty json {}
> -----------------------------------------------
>
> Key: STORM-2568
> URL: https://issues.apache.org/jira/browse/STORM-2568
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 1.1.0
> Reporter: Haewan Jeong
>
> Hello
> I've tried to use storm-kafka-monitor, and it works fine on command line If I
> changed 'toollib/storm-kafka-monitor-*.jar' to
> 'toollib/storm-kafka-monitor-1.1.0.jar'.
> {code}
> {"my-kafka-topic-name":{"0":{"consumerCommittedOffset": 74804998,
> "logHeadOffset": 74805483, "lag": 485},"1":{"consumerCommittedOffset":
> 74804998, "logHeadOffset": 74805485, "lag":
> 487},"2":{"consumerCommittedOffset": 74804995, "logHeadOffset": 74805485,
> "lag": 490},"3":{"consumerCommittedOffset": 74805001, "logHeadOffset":
> 74805488, "lag": 487},"4":{"consumerCommittedOffset": 74805011,
> "logHeadOffset": 74805484, "lag": 473},"5":{"consumerCommittedOffset":
> 74805009, "logHeadOffset": 74805485, "lag":
> 476},"6":{"consumerCommittedOffset": 74805008, "logHeadOffset": 74805483,
> "lag": 475},"7":{"consumerCommittedOffset": 74805010, "logHeadOffset":
> 74805484, "lag": 474},"8":{"consumerCommittedOffset": 73641446,
> "logHeadOffset": 74805488, "lag": 1164042},"9":{"consumerCommittedOffset":
> 73641448, "logHeadOffset": 74805489, "lag":
> 1164041},"10":{"consumerCommittedOffset": 73641443, "logHeadOffset":
> 74805483, "lag": 1164040},"11":{"consumerCommittedOffset": 73641445,
> "logHeadOffset": 74805487, "lag": 1164042},"12":{"consumerCommittedOffset":
> 74805003, "logHeadOffset": 74805486, "lag":
> 483},"13":{"consumerCommittedOffset": 74804999, "logHeadOffset": 74805482,
> "lag": 483},"14":{"consumerCommittedOffset": 74805002, "logHeadOffset":
> 74805483, "lag": 481},"15":{"consumerCommittedOffset": 74805002,
> "logHeadOffset": 74805484, "lag": 482},"16":{"consumerCommittedOffset":
> 74804994, "logHeadOffset": 74805482, "lag":
> 488},"17":{"consumerCommittedOffset": 74805002, "logHeadOffset": 74805489,
> "lag": 487},"18":{"consumerCommittedOffset": 74805003, "logHeadOffset":
> 74805488, "lag": 485},"19":{"consumerCommittedOffset": 74805003,
> "logHeadOffset": 74805489, "lag": 486}}}
> {code}
> but it gives empty result when I call below api.
> {code}
> /api/v1/topology/:id/lag
> ...
> {
> "MySpoutName": {
> "spoutLagResult": {},
> "spoutId": "MySpoutName",
> "spoutType": "KAFKA"
> }
> }
> {code}
> -I think that needs to fix "groupid" to "group.id" in TopologySpoutLag.java I
> debug it, but groupid is right.-
> the reason was topics has square brackets in command.
> {code}
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] json
> configuration: {config.security.protocol=null,
> config.bootstrap.servers=kafka.xxx.com:9092, config.topics=[my-kafka-topic],
> config.groupid=my-storm-kafka-spout-groupid, topology.tasks=5}
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
> /my/program/storm/bin/storm-kafka-monitor
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -t
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
> [my-kafka-topic]
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -g
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
> my-storm-kafka-spout-groupid
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -b
> 2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO]
> kafka.xxx.com:9092
> {code}
> the square brackets automatically added because of this
> {code}
> package org.apache.storm.kafka.spout;
> public class NamedSubscription extends Subscription {
> ...
> @Override
> public String getTopicsString() {
> return String.valueOf(topics);
> }
> {code}
> topics is Collections. so String.valueOf returns value with square brackets.
> I fixed the code that remove square brackets in TopologySpoutLag.java. but I
> think that fixing 'getTopicsString of NamedSubscription.java in
> org.apache.storm.kafka.spout' is might be better.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
