Hello Patrick,
On Aug 16, 2011, at 9:06 , Patrick Ohly wrote:
>> On Aug 3, 2011, at 16:31 , Patrick Ohly wrote:
>>
>>> [...] Is there some way to force UID comparison for added items even in a
>>> normal, incremental sync?
>>
>> Something like a slow sync match on UIDs for adds, to convert them to
>> updates?
>
> Right.
>
>> That's something that could be added to the engine if we add UID-based
>> matching support natively (would make sense, but also quite some work
>> to get it right).
>
> Would it be simpler if it was done without native UID-based matching?
>
> Perhaps by specifying that a comparescript also needs to be run for
> adds?
It would be a run trough the entire sync set. Which is never available in a
searchable form in the client side (and I don't have a reasonably quick and
safe idea how to change that), and in the server it is only loaded in slow
syncs - during normal syncs only changes are loaded. Of course that could be
changed, but I doubt it's worth the effort - because getting it right without
breaking some of the more complicated features like filtering will be hard
work...
>> However I once had a similar case, where a backend would do an
>> implicit merge instead of an add based on the contents (wasn't the
>> UID, but that doesn't matter) of the to-be-added entry.
>>
>> For that we added the possibility for the plugin to return
>> DB_DataMerged (207) when adding the item caused some sort of merge in
>> the backend.
>
> SyncEvolution already does that. But because it can't do a real merge of
> the data, some information is lost.
Why can't it do a real merge? If I understood the scenario correctly, at the
time the "other" item arrives from the peer the backend still has it's own
version unmodified, and should be able to merge both semantically correctly
behind the scene, and then mark one deleted and the other one changed for the
next sync?
I can imagine imagine practical reasons which make this difficult/not feasible
to do in your backend without duplicating a lot of functionality for doing the
merge (hello, libvxxx...) but otherwise I see no obstacle.
>> This causes the engine to read the item back from the backend after
>> having added it, assuming to get back the merged data, which is then
>> sent back to the client in the same session.
>> This only works server-side, because on the client there is no way to
>> send updates after having received server updates (must wait for the
>> next session).
>>
>> Your backend would need to issue a delete for the other item in a later
>> session to clean up.
>
> SyncEvolution is not doing that. Can you elaborate?
>
> What happens is this, from the backend's perspective:
> * Item with local ID FOO exists.
> * Backend asked to add new item.
> * Backend detects that the new item is the same as the existing
> item, returns 207 and local ID FOO.
> * In the next sync, the item with ID FOO is reported as
> "unchanged". Nothing is said about the other item because as far
> as the backend is concerned, it doesn't exist and never has.
But yes, when the sync started, the backend should have reported the original
item as a new one (before the merge occurred). So for the engine, the situation
would be that
* item with localID FOO exists
* item with remotedID BAR is coming in as an add
* backend merges it with existing item and returns localID FOO with
status 207
Now in the server case, the merge occurs BEFORE changes are sent to the client.
So the server should probably
* read back the item with localID FOO to get the result of
the merge (that's what actually happens already)
* search the list of items to-be-sent to the client for
another item with the same localID. That would be the
item reported as an add to the engine BEFORE the merge
* if one is found, and it is an add, remove that item from
the to-be-sent list to avoid to ever create a duplicate
at the client's end
* if one is found, and it is a replace for a already mapped
remoteID, convert it to a delete. After all, it could
well be that the sequence of events is:
* invitation added to server
* sync -> new item gets added to client
* invitation added to client (dumb, not detecting
duplicate)
* sync -> client side version gets added to server,
detected as duplicate.
All but the first bullet point are not implemented so far.
>> Overall, I'm not sure where these types of cross-item merges belong
>> conceptually. I guess with series and exceptions such a merge could
>> easily get more complicated and involve multiple items, so I wonder if
>> that should better be implemented outside the SyncML scope, like a
>> "robot user" who is editing the database between syncs on one end or
>> the other.
>
> I think it fits into the engine. The Funambol server has worked in this
> mode (always do mandatory duplicate checking on adds) for a long time. I
> have my doubts whether it is always the right choice (for example, one
> cannot add "similar" contacts that the server deems identical), but for
> iCalendar 2.0 UID it would be.
I see that the actions to be taken after a backend-detected merge make sense to
be added to the engine, as outline above.
However, actually detecting and merging a duplicate belongs into the backend,
as the search usually must extend beyond what the engine sees as "current sync
set" (imagine a date-range filtered sync, and an invitation added on both sides
which is to far in the future to be in the date range window. The candidate for
a merge could not be found in the sync set!).
> The problematic part is indeed the client side, because a duplicate
> detected there will temporarily leave client and server out of sync. But
> when assuming that the server does the same duplicate checking, it can
> only happen when a new item was added on the client during the sync
> (otherwise the server would have detected the duplicate).
Agreed. Technically, modifications happening *during* sync in real time are
considered having happened *after* sync (except if the backend cannot pinpoint
the modification dates of all entries modified by sync excactly to the sync
reference token sampled at the beginning of the sync).
Best Regards,
Lukas Zeller
[email protected]
http://www.plan44.ch/syncmlios.php
_______________________________________________
os-libsynthesis mailing list
[email protected]
http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
