On 30.4.14 9:41 , Davide Giannella wrote:
On 29/04/2014 16:06, Davide Giannella wrote:
...
Serving searches could be done either by the `:next` stream or by the
`:next` and then merged on the fly with the current `:next-ID` in the
iterator while returning the resultset.
Any thoughts on the above?
More thinking around it... if can't find a way to deliver merged
searches on the fly it doesn't worth the effort as it will work as an
asynchronous index with some extra complexity.
Another idea that came to mind, which might be easier to implement and
require fewer changes to the existing code:
Stay with the linked list / skip list approach but defer inserting new
keys to a background operation. Instead new keys are put into a
"pending-non-conflicting-id" container until picked up by a background
thread and merged into the linked list. Conflicts are simply handled by
a first committer wins strategy in the same way we handle the entire
index update currently.
When such an index is used, the "pending-non-conflicting-id" container
would have to be considered and its content would have to be merged into
the linked list on the fly. This should be rather easy by first sorting
these extra keys and then on the fly merging the linked list and the
sorted extra keys.
I think such an approach is traded off very well: the index is always
consistent, however depending on the number of keys in the
"pending-non-conflicting-id" container performance might degrade a bit
until the background process has caught up again. Additionally this
approach is only slightly more complicated than what we already have.
Michael
