Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1568904980 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh merged PR #16190: URL: https://github.com/apache/druid/pull/16190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1567570396 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

github-advanced-security[bot] commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566594802 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -122,6 +126,7 @@

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566576116 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaSeekableStreamEndSequenceNumbers.java: ## @@ -0,0 +1,183 @@ +/* + *

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566575886 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566575647 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaSeekableStreamStartSequenceNumbers.java: ## @@ -0,0 +1,220 @@ +/* + *

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566575482 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaSeekableStreamStartSequenceNumbers.java: ## @@ -0,0 +1,220 @@ +/* + *

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566574921 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566575304 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1566575090 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

abhishekrb19 commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1562694558 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1563017112 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

abhishekrb19 commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1562793256 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

abhishekrb19 commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1562674637 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1557907260 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1554346513 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaSeekableStreamStartSequenceNumbers.java: ## @@ -0,0 +1,158 @@ +/* + *

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

github-advanced-security[bot] commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1548924208 ## extensions-core/kafka-indexing-service/src/test/java/org/apache/druid/indexing/kafka/KafkaDataSourceMetadataTest.java: ## @@ -203,19 +462,60

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1548910010 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaSeekableStreamStartSequenceNumbers.java: ## @@ -0,0 +1,158 @@ +/* + *

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1548909835 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaSeekableStreamEndSequenceNumbers.java: ## @@ -0,0 +1,126 @@ +/* + *

Re: [PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

abhishekagarwal87 commented on code in PR #16190: URL: https://github.com/apache/druid/pull/16190#discussion_r1545980952 ## extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java: ## @@ -444,4 +447,56 @@ public

[PR] Kafka with topicPattern can ignore old offsets spuriously (druid)

zachjsh opened a new pull request, #16190: URL: https://github.com/apache/druid/pull/16190 Fixes #16189 ### Description This fixes an issue in which updating a kafka streaming supervisors topic from single to multi-topic (pattern), or vice versa, could cause old offsets to