[
https://issues.apache.org/jira/browse/OAK-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031319#comment-16031319
]
Stefan Egli commented on OAK-6276:
----------------------------------
bq. I was just wondering if we can have some kind of assumptions and help in
putting a lower bound on what we want to do here
sure. I was assuming that there would be no correlation between which instance
produced messages and in what order any message (from the same or different
instance producers) will be consumed, thus end up being compared using this new
API.
bq. if we assume that all message must always get processed serially
that's not the case no. the messages might in some cases be processed serially,
sometimes in a completely different order. I think the order should be seen as
random.
bq. What I meant NodeStore API has no semantics of ordering of revision
Ok, which is probably a good argument why this shouldn't be leaked outside of
Oak with any such new API.
bq. unless you assume that X,Y,Z in [rX-0-1, rY-0-2,rZ-0-3] are timestamps
Ok, agreed. Upper level code should not go that far as to having to do any
interpretation on these revisions. But the DocumentNodeStore itself can..
bq. But, along with that, we also need to assert some way that whatever token
thing is passed along can still work with revision that could be re-written on
a RevGC – I haven't followed with current work on document-node-store gc by
Marcel... but seg-store, for example, would "lose" a revision that can be
peeked into after an compaction cycle..
Good point. So even though you point out later that this should be ignored :)
it's I think an important point to keep in mind.
This could be seen then as a requirement to eg add a generation-id-like prefix
to such comparable checkpoint like strings. Such that when it compacts and
later has to do a comparison with an old generation 'checkpoint' it could
safely regard it as 'old'.
Which is also why - perhaps - it is easier to limit this functionality to only
ever compare some old 'checkpoint' with the current head - and *not* with an
arbitrary other such 'checkpoint' (which in the case of compaction becomes
impossible to implement).
Which is - further - why perhaps it's simpler to limit getting this
'checkpoint-lite' only from the current head and not on an (arbitrary old)
session.
bq. I understand that we don't plan to support seg-store
I think it would be good if we would either support the segment node store as
well - or at least have a good understanding of *how* it could be supported
(and it should be fast and simple). If we find something that would block it
from being supported, then I think it might be a sign for a bad API.
Not sure what would be the easiest for segment node store as ordering by
segments doesn't seem to do the trick I guess..
> expose way to detect "eventual consistency delay"
> -------------------------------------------------
>
> Key: OAK-6276
> URL: https://issues.apache.org/jira/browse/OAK-6276
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api
> Reporter: Stefan Egli
>
> I have a requirement to support an external messaging channel (eg Kafka)
> between Oak-based instances of the same cluster. As part of handling those
> messages the target instance in some cases might have to access data from the
> repository.
> Now with DocumentNodeStore's eventual consistency that data might not
> 'travel' from the source to the target instance as fast as is the case for
> the external message.
> Therefore the need arises to be able to delay such messages (on the target
> instance) until the repository sees (at least) the data the source instance
> wrote when sending off the message.
> This ticket is to equip Oak with any feasible way for higher level code to
> generally speaking detect such an "eventual consistency delay".
> One simple idea that comes to mind is to expose the current _head revision
> vector_ (or that from a particular session, but that might not be required,
> ie be too complicated). The source instance could get the local head revision
> vector, piggyback that on the message, then that could be compared on the
> target instance with its head state. If that turns out to be older, then it
> could do a wait and retry. (Nicer would of course be if there would even be a
> call-back - but in theory that could also be implemented ontop of an
> Observer).
> One means to expose the head revision vector would be via a repository
> descriptor (which on access returns the current value, similar to [how
> discovery-lite does
> it|https://github.com/apache/jackrabbit-oak/blob/2634dbde9aedc2549f0512285e9abee5858b256f/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentDiscoveryLiteService.java#L246]).
> And the format could be normalized as eg {{longs}} (eg {{\[1496071927014,
> 1496071926243]}} (instead of {{\["r15c54d532ec-0-1", "r15c54d532ec-0-2"]}} to
> avoid leaking the revision format explicitly).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
