On 06/05/2014 20:29, Michael Dürig wrote:
> On 6.5.14 2:59 , Davide Giannella wrote:
>> On 02/05/2014 10:38, Michael Dürig wrote:
>>> Stay with the linked list / skip list approach but defer inserting new
>>> keys to a background operation. Instead new keys are put into a
>>> "pending-non-conflicting-id" container until picked up by a background
>>> thread and merged into the linked list. Conflicts are simply handled
>>> by a first committer wins strategy in the same way we handle the
>>> entire index update currently.
>>>
>>> When such an index is used, the "pending-non-conflicting-id" container
>>> would have to be considered and its content would have to be merged
>>> into the linked list on the fly. This should be rather easy by first
>>> sorting these extra keys and then on the fly merging the linked list
>>> and the sorted extra keys.
>> ...
> ...
Had a long off-line discussion about the topic with Michael where we
brainstormed structures, use-cases and conflicts.
We're going to go for one of the following:
# Solution 1
:index : {
_pending : {
prop-550e8400-e29b-41d4-a716-446655440000 : {
foo="/path/to/the/content" },
prop-550e8400-e29b-41d4-a716-446655440000 : {
bar="-/path/to/the/content" } <== deletion
}
}
where each cluster node/session add a
prop-UUID : { $property-value = "${operation-if-needed}/pathtocontent" }
then the async merger will parse those, apply to the actual structure
and delete from the _pending area.
## Pros
definitely avoid conflicts
works as a sort of ChangeLog
## Cons
will be harder the merging of results.
# Solution 2
:index : {
keyA : {
path : { to : { content : {match=true} } }
},
keyB : {
path : { to : { content : {deleted=true} } }
}
}
where each cluster use the _pending area as it would be a real index (no
order in here though) and will never physically delete a node.
The async merger will parse and merge into the actual index cleaning-up
while doing it.
## Pros
Easy for on-the-fly merging while serving queries
## Cons
Could not avoid conflict as we expect.
Will update the ticket accordingly for the records.
Cheers
Davide
