[
https://issues.apache.org/jira/browse/KAFKA-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251690#comment-17251690
]
Bruno Cadonna edited comment on KAFKA-10843 at 12/18/20, 11:02 AM:
-------------------------------------------------------------------
I had another look at the code and I see two options {{metadataForKey()}} might
return {{null}}.
1. There are no source topics in the topology for the given state store (see
[code|https://github.com/apache/kafka/blob/0efa8fb0f4c73d92b6e55a112fa45417a67a7dc2/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java#L165]).
2. Kafka Streams finds the wrong source topic partition for the given key (see
[code|https://github.com/apache/kafka/blob/0efa8fb0f4c73d92b6e55a112fa45417a67a7dc2/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java#L411]).
I guess, option 1 does not apply since the same topology worked in your other
environments.
Is your data partitioned differently in your PROD environment than in your
other environments? Do you use a custom partitioner for the data in your PROD
environment?
was (Author: cadonna):
I had another look at the code and I see two options {{metadataForKey()}} might
return {{null}}.
1. There are no source topics in the topology for the given state store (see
[code|https://github.com/apache/kafka/blob/0efa8fb0f4c73d92b6e55a112fa45417a67a7dc2/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java#L165]).
2. Kafka Streams finds the wrong source topic partition for the given key (see
[code|https://github.com/apache/kafka/blob/0efa8fb0f4c73d92b6e55a112fa45417a67a7dc2/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java#L411]).
I guess, option 1 does not apply since the same topology worked in your other
environments.
Is your data partitioned differently in your PROD environment than in your
other environments?
> Kafka Streams metadataForKey method returns null but allMetadata has the
> details
> --------------------------------------------------------------------------------
>
> Key: KAFKA-10843
> URL: https://issues.apache.org/jira/browse/KAFKA-10843
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.5.1
> Reporter: Maria Thomas
> Priority: Major
>
> Our application runs on multiple instances and to enable us to use get the
> key information from the state store we use "metadataForKey" method to
> retrieve the StreamMetadata and using the hostname do an RPC call to the host
> to get the value associated with the key.
> This call was working fine in our DEV and TEST environments, however, it is
> failing one our production clusters from the start. On further debugging, I
> noticed the allMetadata() method was returning the state stores with the host
> details as expected. However, it would not be feasible to go through each
> store explicitly to get the key details.
> To note, the cluster I am using is a stretch cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
