Am Mi., 2. Jan. 2019 um 02:05 Uhr schrieb Timothee Maret <[email protected] >:
> Hi, > > I looked at the API considering how we could use it for our replication use > case. I identified one key concept that seemed to be missing, the indexing > of messages with monotonically increasing offsets. > > For replication, we leverage those offsets extensively, for instance to > efficiently compute sub ranges of messages, to skip range of messages, to > delay processing of messages, to clean up resources, etc. If we want to > leverage the journaled event API to guarantee portability, it seems to me > that we'd need to have the offset or an equivalent construct part of the > API. > > How about adding a "getOffset" signature and documenting the offset > requirement in the Position interface ? > I just started implementing the in memory impl of the API and also used an offset. For the cases I know an offset makes sense. Alexei and I were just unsure if the offset is really a general abstraction. If we all agree an offset makes sense then I am in favour of adding it. Actually in the case of no partitions (wich we currently assume) the position is not more than an offset. > Another unclear intention to me in the API, is the support for partitions > (similar to Kafka). The documentation indicates it is not a goal, however > the API seems to contain some hints for multi-partition support such as the > "TopicPosition" interface. How about supporting multiple partitions in the > API by allowing to specify a key (with a semantic similar to Kafka) in the > "newMessage" signature ? > I removed the TopicPosition interface again a few days ago. It was not part of the API Alexei and I discussed and makes no sense when we limit ourself to no partitions (or 1 partition in case of kafka). So the main question is if limiting ourselves is a good idea. I think it is but I would be very interested in other opinions. Cheers Christian -- -- Christian Schneider http://www.liquid-reality.de Computer Scientist http://www.adobe.com
