gianm commented on issue #6431: Add Kinesis Indexing Service to core Druid URL: https://github.com/apache/incubator-druid/pull/6431#issuecomment-428032687 > First when you read the code you can not really tell what is the semantic around T1 and T2 ? can this be any object really? or does it have to be comparable ? or it extend numbers ? IMO we need to clear that out. is it really true that every record is part of a partition and has a sequence number? I haven't read the patch yet, but I think it is useful to have an abstraction for ordered-partitionable streams (like Kafka and Kinesis have). It's because we have a lot of logic specifically around dealing with those kinds of systems when it comes to partition discovery, task creation, work assignment, and metadata handling. A lot of that shared code won't apply if the streams do not have an ordered-partition concept. It seems like Pulsar topics are ordered-partitionable (they can have partitions, and those partitions have messages with ordered sequence IDs) so it should be able to fall under the same abstraction. I am not familiar with Pravega so I don't know about that one.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
