jhgg commented on issue #9343: URL: https://github.com/apache/druid/issues/9343#issuecomment-686811976
GCP has recently introduced a PubSub variant that roughly operates like kafka, called PubSub Lite - wherein you have a topic, and you have a bunch of partitions, and you consume from the partitions with a given offset. https://cloud.google.com/pubsub/docs/choosing-pubsub-or-lite > Lite topics consist of partitions; a partition is a log of messages. Publishers assign each message to a partition, and the Pub/Sub Lite service appends each message to the log. The Pub/Sub Lite service delivers the messages from a partition in order. Given this very close model to Kafka, I'm thinking that a PubSub Lite indexing service would actually be fairly straightforward to implement. Discord would like to fund these efforts, if anyone wants to take this on. Feel free to e-mail me [email protected] (or reply here if that's OK) if you're interested. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
