[
https://issues.apache.org/jira/browse/METRON-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505977#comment-16505977
]
ASF GitHub Bot commented on METRON-1573:
----------------------------------------
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1030
> [Stellar]>>> KAFKA_TAIL("mytopic")
> []
KAFKA_TAIL always seeks to the end of the topic. So the consumer offset
was set to the end of the topic (offset=2). No more messages arrived on this
topic, so it timed out and returned no data.
If you had run this on a topic actively receiving data, you would have seen
the new messages arriving.
> [Stellar]>>> KAFKA_GET("mytopic")
> []
Like I said, after running the KAFKA_TAIL command before your consumer
offset was set to 2 (the end of the topic.) KAFKA_GET will always use an
existing consumer offset. Since there are no messages at offset 2, It didn't
return anything.
Use KAFKA_TAIL on one of the sensor topics in the dev environment. You
will be able to see the latest messages arriving.
```
KAFKA_TAIL("bro")
```
You can also manually change your `group.id` so that your KAFKA_GET call
doesn't pick-up the existing consumer offsets, if that is what you want.
```
KAFKA_GET("mytopic",{ 'group.id':'new.group' })
```
> Enhance KAFKA_* functions to return partition and offset details
> ----------------------------------------------------------------
>
> Key: METRON-1573
> URL: https://issues.apache.org/jira/browse/METRON-1573
> Project: Metron
> Issue Type: Improvement
> Reporter: Nick Allen
> Assignee: Nick Allen
> Priority: Major
>
> The KAFKA_* functions currently simply return the value of the message.
> There are often times when it would be useful to get more detailed
> information including the message partition, offset, key, etc.
> The functions should be enhanced to allow a user to optionally get a more
> detailed view.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
