Hello! I'm trying to batch local database adds together to increase performance in SyncEvolution (https://bugs.freedesktop.org/show_bug.cgi?id=52669 and https://bugs.freedesktop.org/show_bug.cgi?id=64838#c5).
My hope was that I can postpone the actual modification and return a temporary local ID. Then later or in parallel, combine multiple items together in a batch add. Finally, the temporary ID was meant to be replaced by the final one in FinalizeLocalID. But I hit the first snag pretty early: FinalizeLocalID() doesn't seem to get called on the server side where I need this feature primarily (for PBAP, CalDAV, ActiveSync the server side is the one with the local database). dsFinalizeLocalID, the wrapper around the store's FinalizeLocalID, gets called in three functions: (gdb) b sysync::TBinfileImplDS::changeLogPostflight Breakpoint 2 at 0x106f956: file /home/pohly/syncevolution/libsynthesis/src/sysync/binfileimplds.cpp, line 462. (gdb) b sysync::TBinfileImplDS::SaveAdminData Breakpoint 3 at 0x10739fd: file /home/pohly/syncevolution/libsynthesis/src/sysync/binfileimplds.cpp, line 2389. (gdb) b sysync::TLocalEngineDS::engGenerateMapItems Breakpoint 4 at 0x109c79c: file /home/pohly/syncevolution/libsynthesis/src/sysync/localengineds.cpp, line 6635. None of these get called on the server side during a refresh-from-client, slow or normal two-way sync. That makes sense, map items are only relevant on the client, and the server is not using the binfile class. But shouldn't the server also call FinalizeLocalID somewhere? Where would be the right place for it? Regarding FinalizeLocalID() in the client, I see some conceptual issues with it, at least for what I am trying to achieve. My main concern is about error handling. FinalizeLocalID() can return an error code, but dsFinalizeLocalID does not check for errors. It merely checks for "ID modified". So in case of an error, the local store itself has to remember that writing failed and "do something". What that "something" could be is not clear. It could return an error in EndDataWrite(), in which case I would expect the engine to enforce a slow sync in the next sync session. I can't think of other, realistic options right now. I think a better solution would be to delay handling the result of a database write until the entire SyncML message has been processed. Then give the store a chance to finalize the ID of each item that was touched as part of that SyncML message. The store can use that opportunity to flush the pending operations and return error codes for each item, just as it would normally do in the synchronous write operations. Then the engine continues with the same error handling that it already does. The advantage, besides better error handling, is that the store has a natural place to flush pending operations. Without the intermediate FinalizeLocalID() calls a new "max pending operations" setting independent of the SyncML message size would be needed. Comments? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. _______________________________________________ os-libsynthesis mailing list os-libsynthesis@synthesis.ch http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
