kaisun2000 opened a new issue, #13994: URL: https://github.com/apache/druid/issues/13994
See the discussion in this slack thread https://apachedruidworkspace.slack.com/archives/C0303FDCZEZ/p1680036508569219?thread_ts=1679951040.402899&cid=C0303FDCZEZ. Overall, if we have replicated ingestion task, due to various reasons, there is a possibility that the same real time query may see result back in time if the query goes to the slower ingestion task the 2nd time. One proposal is that we use the Kafka offset as a hint to the query via query context. The query can be configured to return the Kafka offsets corresponding to all the Kafka partitions. These offsets can be also used as a hint via query context to the peons serving the query. The peon should only serve the query once its ingestion passed all the offsets of the partitions the peon is responsible for. This would eliminate the potential inconsistent result for realtime query. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
