wcarlson5 commented on a change in pull request #11847:
URL: https://github.com/apache/kafka/pull/11847#discussion_r819814891
##########
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java
##########
@@ -1259,13 +1259,15 @@ public OffsetResetStrategy offsetResetStrategy(final
String topic) {
} else if
(maybeDecorateInternalSourceTopics(latestResetTopics).contains(topic) ||
latestResetPatterns.stream().anyMatch(p ->
p.matcher(topic).matches())) {
return LATEST;
- } else if
(maybeDecorateInternalSourceTopics(rawSourceTopicNames).contains(topic)
- || !hasNamedTopology()
- || (usesPatternSubscription() &&
Pattern.compile(sourceTopicPatternString).matcher(topic).matches())) {
+ } else if (fullSourceTopicNames().contains(topic)
+ || (usesPatternSubscription() &&
Pattern.compile(sourceTopicPatternString()).matcher(topic).matches())) {
return NONE;
} else {
- // return null if the topic wasn't found at all while using
NamedTopologies as it's likely in another
- return null;
Review comment:
Much better than returning null
##########
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java
##########
@@ -978,24 +979,30 @@ private void resetOffsets(final Set<TopicPartition>
partitions, final Exception
final Set<TopicPartition> notReset = new HashSet<>();
for (final TopicPartition partition : partitions) {
- switch (topologyMetadata.offsetResetStrategy(partition.topic())) {
- case EARLIEST:
- addToResetList(partition, seekToBeginning, "Setting topic
'{}' to consume from {} offset", "earliest", loggedTopics);
- break;
- case LATEST:
- addToResetList(partition, seekToEnd, "Setting topic '{}'
to consume from {} offset", "latest", loggedTopics);
- break;
- case NONE:
- if ("earliest".equals(originalReset)) {
- addToResetList(partition, seekToBeginning, "No custom
setting defined for topic '{}' using original config '{}' for offset reset",
"earliest", loggedTopics);
- } else if ("latest".equals(originalReset)) {
- addToResetList(partition, seekToEnd, "No custom
setting defined for topic '{}' using original config '{}' for offset reset",
"latest", loggedTopics);
- } else {
- notReset.add(partition);
- }
- break;
- default:
- throw new IllegalStateException("Unable to locate topic "
+ partition.topic() + " in the topology");
+ final OffsetResetStrategy offsetResetStrategy =
topologyMetadata.offsetResetStrategy(partition.topic());
+
+ // This may be null if the task we are currently processing was
apart of a named topology that was just emoved.
+ // TODO: keep the StreamThreads and TopologyMetadtaa view of named
topologies in sync until final thread has acked
+ if (offsetResetStrategy != null) {
Review comment:
Is this null check still necessary as we don't return null anymore? (and
maybe we can update the `offsetResetStrategy`) as well
This can be a follow up as it is not going to change the behavior and it
would be good to get this fix for a flaky test it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
