Hi!
Some mails didn't make it to the list because the email address got
replaced. Resending...
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--- Begin Message ---
Lukas, thanks for your reply and your work!
I am still not definite on your message, I will repeat here hoping you can give
me further clarification.
Regarding the first part of your message, do you mean SyncEvolution doesn't have
to change tracking MORE THAN ONE STATE per source since it is built with
BASED_ON_BINFILE_CLIENT? If this is true, the current implementation in
synthesis client is CORRECT, why do you think this is a conceptual bug?
In my own understanding, the change log maintained by binfile glue layer is the
synthesis client specific information (i.e. which items has been successfully
sent, which items was not yet sent out or failed during sent out); while the
change log maintained by SyncEvolution (DBplugin) is db specific information
(i.e. which items has been changed since last time synthesis client asked for).
Coming back to the suspend/resume support, assuming last sync is suspended.
The question is: During next sync, does the synthesis client need *only* changes
since last time her asked for SyncEvolution(last suspended sync)? Or it needs
*both* changes since last suspended sync and changes since last successful sync?
If the former is true, SyncEvolution only needs to change tracking *One* state;
if the later is true, SyncEvolution still needs to change tracking *Two* states.
The spec says: After the suspended session, both the client and the server can
refues a resume, in that case a slow sync *must* be issued. This means during
the
next sync (after a suspended session), either a resuming session is issued or a
slow sync. Therefore the server only needs changes before last suspended sync
for
resume! (Otherwise, during a slow sync, all the data will be sent to the server,
no need for any change log).
Thanks!
On Sat, 13 Jun 2009, Lukas Zeller wrote:
> Hello Chen,
>
> Sorry it took me a while to respond. I had to review the
> implementation which is a few years old now to grasp the details again.
>
> Please also apologize that I only now realize that when discussing bug
> 2425[1] for syncEvolution with Patrick, I was probably not thinking
> precisely enough and may have caused wrong conclusions as what is
> needed to do in SyncEvolution.
>
> The most important fact is that the change detection in the case of
> BASED_ON_BINFILE_CLIENT (which is how libsynthesis is built) is a two-
> stage process.
>
> There are other configurations of the engine that don't use the
> binfileimplXXX implementation, and thus have a one-stage change
> detection. On the other hand, the DB plugin API is designed such that
> it can also be used without the intermediate stage of binfileimplXXX,
> so some of what the API exposes (in particular: StartDataRead's
> resumeToken argument) is not used in an engine built with
> BASED_ON_BINFILE_CLIENT.
>
> The change detection is two stage because binfileimplXXXX was designed
> as an intermediate layer for backends that have no usable change
> detection by its own at all (such as PalmOS) or cannot pinpoint
> changes on a time scale (just have a "dirty" flag).
>
> So what binfileimpXXX does (should do, see bug discussion below) is:
>
> 1) for every sync it asks the DB to report all items, along with a
> flag indicating which items have changed SINCE THE LAST TIME IT ASKED
> FOR CHANGES (which is not necessarily the last sync).
>
> 2) With this information, a separate changelog (as *_clg.bfi file) is
> maintained. This changelog (which is also shared between multiple
> profiles, i.e. multiple sync servers syncing the same datastore) saves
> a modification count for each item, which allows to pinpoint changes
> on a time scale equivalent (the modCount scale).
>
> 3) The "last sync" and "last suspend" points are internally saved in
> terms of modCounts. All decisions if a item needs to be resent/sent in
> a sync/resume are based on these modCount comparisons.
>
> So the bottom line for DB plugins used exclusively with an synthesis
> engines built with BASED_ON_BINFILE_CLIENT (as the current
> libsynthesis) is that they DO NOT NEED TO TRACK MORE THAN ONE CHANGE
> STATE:
>
> - StartDataRead() will always be called with only one token
>
> - EndDataWrite() will be called at end of sync and for suspend, such
> that the next run of ReadNextItem() will only report changes happened
> since the last run.
>
> Sorry Patrick for saying something different (relating to non-
> binfileXXX datastores, and wrong in libsynthesis context) in our
> earlier discussion.
>
>
> So far the theory. Obviously, there's something wrong in the
> implementation regarding the tokens passed to StartDataRead(), and
> calling endDataWrite(), as you already found out. I suspect that the
> BASED_ON_BINFILE_CLIENT glue implementation is based on a (my own)
> confusion between the two layers of changelogs I explained above.
>
> Thanks to your investigation, I think I know where to look for that
> conceptual bug. I guess your fix already does the right thing for the
> BASED_ON_BINFILE_CLIENT case, but I'll have to check if it works as
> well in the other build variants. I'll have a look and post your fix
> or a modified version upstream as soon as possible.
>
> Best Regards,
>
> On Jun 12, 2009, at 6:48 , Chen Congwu wrote:
>
> > Hello
> >
> > It is expected the token returned in EndDataWrite should be stored
> > by synthesis
> > and return this value in StartDataRead during next session. This
> > should be true
> > both for successful sync and suspended sync.
> >
> > However currently if the sync is suspended, the token will not be
> > stored (when
> > build with BASED_ON_BINFILE_CLIENT on). There are two issues causing
> > this
> > behaviour:
> > 1) No write to fPreviousSuspendIdentifier
> > 2) EndDataWrite should be called before engSaveSuspendState.
> >
> > The following violent workaround make it works as I expected, I
> > attached here
> > maybe help you to understand my email.
> >
> > diff --git a/src/sysync/binfileimplds.cpp b/src/sysync/
> > binfileimplds.cpp
> > index 5ea5a7a..448d153 100755
> > --- a/src/sysync/binfileimplds.cpp
> > +++ b/src/sysync/binfileimplds.cpp
> > @@ -1832,6 +1832,8 @@ localstatus
> > TBinfileImplDS::implSaveResumeMarks(void)
> > {
> > // update modcount reference of last suspend
> > fPreviousSuspendModCount = fCurrentModCount;
> > +
> > + fPreviousSuspendIdentifier = fCurrentSyncIdentifier;
> > // save admin data now
> > return SaveAdminData(true,false); // end of session, but not
> > successful
> > } // TBinfileImplDS::implSaveResumeMarks
> > diff --git a/src/sysync/stdlogicds.h b/src/sysync/stdlogicds.h
> > index 5271e7e..56a3d9c 100755
> > --- a/src/sysync/stdlogicds.h
> > +++ b/src/sysync/stdlogicds.h
> > @@ -148,7 +148,7 @@ protected:
> > /// markOnlyUngeneratedForResume(), markItemForResume() and
> > markItemForResend())
> > /// (or, in case the session is really complete, make sure that no
> > resume state is left)
> > /// @note Must also save tempGUIDs (for server) and pending/
> > unconfirmed maps (for client)
> > - virtual localstatus logicSaveResumeMarks(void) { return
> > implSaveResumeMarks(); };
> > + virtual localstatus logicSaveResumeMarks(void) { endDataWrite();
> > return implSaveResumeMarks(); };
> >
> > --
> > Regards,
> > Chen Congwu
> > Moblin China Development
> >
> >
> > os-libsynthesis mailing list
> > [email protected]
> > http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
>
> Lukas Zeller ([email protected])
> -
> Synthesis AG, SyncML Solutions & Sustainable Software Concepts
> [email protected], http://www.synthesis.ch
>
>
>
--
Regards,
Chen Congwu
Moblin China Development
--- End Message ---
--- Begin Message ---
Hello Chen,
On Jun 13, 2009, at 19:24 , Chen Congwu wrote:
> I am still not definite on your message, I will repeat here hoping
> you can give
> me further clarification.
>
> Regarding the first part of your message, do you mean SyncEvolution
> doesn't have
> to change tracking MORE THAN ONE STATE per source since it is built
> with
> BASED_ON_BINFILE_CLIENT? If this is true, the current implementation
> in
> synthesis client is CORRECT, why do you think this is a conceptual
> bug?
Maybe conceptual was not the correct word. The concept is ok, but I
guess I had some confusion between the DB side and the engine side
concept and believe there IS a problem insofar that the engine does
not always call StartDataRead() with the correct tokens. As said, I
have to check the details.
> In my own understanding, the change log maintained by binfile glue
> layer is the
> synthesis client specific information (i.e. which items has been
> successfully
> sent, which items was not yet sent out or failed during sent out);
> while the
> change log maintained by SyncEvolution (DBplugin) is db specific
> information
> (i.e. which items has been changed since last time synthesis client
> asked for).
Agreed.
> Coming back to the suspend/resume support, assuming last sync is
> suspended.
> The question is: During next sync, does the synthesis client need
> *only* changes
> since last time her asked for SyncEvolution(last suspended sync)?
Yes.
> Or it needs *both* changes since last suspended sync and changes
> since last successful sync?
No - not when built as BASED_ON_BINFILE_CLIENT. StartDataRead() has
two tokens ONLY for the case when the engine is NOT built as
BASED_ON_BINFILE_CLIENT, which is the case for all our servers and the
ODBC client. In these cases, the DB layer is responsible for all
change tracking. With BASED_ON_BINFILE_CLIENT, the change tracking
requirements for the DB is much simpler.
> If the former is true, SyncEvolution only needs to change tracking
> *One* state;
Correct.
> if the later is true, SyncEvolution still needs to change tracking
> *Two* states.
No, altough Patrick and me came to that conclusion in the initial
discussion, it is not needed. The binfileXXX layer makes this
unnecessary.
> The spec says: After the suspended session, both the client and the
> server can
> refuse a resume, in that case a slow sync *must* be issued. This
> means during the
> next sync (after a suspended session), either a resuming session is
> issued or a
> slow sync.
Yes, but that's a new requirement in SyncML DS 1.2.1. In 1.2, this was
not specified, and a server rejecting resume could well request a
normal sync.
I'm not sure why this was changed on 1.2.1. Technically there is IMHO
no reason why a rejected resume must be followed by a slow sync, as
long as the anchors are ok. One guess is that they tried to force
implementors of DS 1.2 servers to actually implement resume and not
fake DS 1.2 compatibility by simply rejecting all resume requests with
509 and then continuing as if it was a normal sync (which a few did).
> Therefore the server only needs changes before last suspended sync for
> resume! (Otherwise, during a slow sync, all the data will be sent to
> the server,
> no need for any change log).
Two notes:
1) Consider the case where a client is synchronizing with more than
one server (all our PRO clients have multiple profiles). For that,
even without suspend&resume, you need a mechanism to pinpoint changes
on a time scale at multiple points, even more than 2 (if you have more
than 2 servers). That what the binfileXXX layer was created for in the
first place.
2) Even in a single server scenario resume, you must be able to
differentiate modifications happened after the original sync and
before the suspend and modifications happened AFTER the suspend. You
are right that this differentiation does not necessarily need a re-
evaluation of the changes since last regular sync as long as the
intermediate layer keeps a list of to-be-resent items somehow.
Best Regards,
Lukas Zeller ([email protected])
-
Synthesis AG, SyncML Solutions & Sustainable Software Concepts
[email protected], http://www.synthesis.ch
--- End Message ---
--- Begin Message ---
On Sun, 2009-06-14 at 17:58 +0100, Lukas Zeller wrote:
> > Therefore the server only needs changes before last suspended sync for
> > resume! (Otherwise, during a slow sync, all the data will be sent to
> > the server,
> > no need for any change log).
>
> Two notes:
>
> 1) Consider the case where a client is synchronizing with more than
> one server (all our PRO clients have multiple profiles). For that,
> even without suspend&resume, you need a mechanism to pinpoint changes
> on a time scale at multiple points, even more than 2 (if you have more
> than 2 servers). That what the binfileXXX layer was created for in the
> first place.
Let me add how SyncEvolution works. It also supports multiple profiles,
called "server configurations". Each configuration has its own set of
Synthesis bin files, so change tracking has to be done per
configuration. All of the current SyncEvolution backends support that,
so suspend&resume shouldn't be affected.
--
Best Regards
Patrick Ohly
Senior Software Engineer
Intel GmbH
Software & Solutions Group
Hermuelheimer Strasse 8a Phone: +49-2232-2090-30
50321 Bruehl Fax: +49-2232-2090-29
Germany
--- End Message ---
--- Begin Message ---
Thanks Lukas, with your help I am now clear about what to do to support
suspend and resume in SyncEvoltion.
Really appreciate!
On Mon, 15 Jun 2009, Lukas Zeller wrote:
> Hello Chen,
>
> On Jun 13, 2009, at 19:24 , Chen Congwu wrote:
>
> > I am still not definite on your message, I will repeat here hoping
> > you can give
> > me further clarification.
> >
> > Regarding the first part of your message, do you mean SyncEvolution
> > doesn't have
> > to change tracking MORE THAN ONE STATE per source since it is built
> > with
> > BASED_ON_BINFILE_CLIENT? If this is true, the current implementation
> > in
> > synthesis client is CORRECT, why do you think this is a conceptual
> > bug?
>
> Maybe conceptual was not the correct word. The concept is ok, but I
> guess I had some confusion between the DB side and the engine side
> concept and believe there IS a problem insofar that the engine does
> not always call StartDataRead() with the correct tokens. As said, I
> have to check the details.
>
> > In my own understanding, the change log maintained by binfile glue
> > layer is the
> > synthesis client specific information (i.e. which items has been
> > successfully
> > sent, which items was not yet sent out or failed during sent out);
> > while the
> > change log maintained by SyncEvolution (DBplugin) is db specific
> > information
> > (i.e. which items has been changed since last time synthesis client
> > asked for).
>
> Agreed.
>
> > Coming back to the suspend/resume support, assuming last sync is
> > suspended.
> > The question is: During next sync, does the synthesis client need
> > *only* changes
> > since last time her asked for SyncEvolution(last suspended sync)?
>
> Yes.
>
> > Or it needs *both* changes since last suspended sync and changes
> > since last successful sync?
>
> No - not when built as BASED_ON_BINFILE_CLIENT. StartDataRead() has
> two tokens ONLY for the case when the engine is NOT built as
> BASED_ON_BINFILE_CLIENT, which is the case for all our servers and the
> ODBC client. In these cases, the DB layer is responsible for all
> change tracking. With BASED_ON_BINFILE_CLIENT, the change tracking
> requirements for the DB is much simpler.
>
> > If the former is true, SyncEvolution only needs to change tracking
> > *One* state;
>
> Correct.
>
> > if the later is true, SyncEvolution still needs to change tracking
> > *Two* states.
>
> No, altough Patrick and me came to that conclusion in the initial
> discussion, it is not needed. The binfileXXX layer makes this
> unnecessary.
>
> > The spec says: After the suspended session, both the client and the
> > server can
> > refuse a resume, in that case a slow sync *must* be issued. This
> > means during the
> > next sync (after a suspended session), either a resuming session is
> > issued or a
> > slow sync.
>
> Yes, but that's a new requirement in SyncML DS 1.2.1. In 1.2, this was
> not specified, and a server rejecting resume could well request a
> normal sync.
> I'm not sure why this was changed on 1.2.1. Technically there is IMHO
> no reason why a rejected resume must be followed by a slow sync, as
> long as the anchors are ok. One guess is that they tried to force
> implementors of DS 1.2 servers to actually implement resume and not
> fake DS 1.2 compatibility by simply rejecting all resume requests with
> 509 and then continuing as if it was a normal sync (which a few did).
>
> > Therefore the server only needs changes before last suspended sync for
> > resume! (Otherwise, during a slow sync, all the data will be sent to
> > the server,
> > no need for any change log).
>
> Two notes:
>
> 1) Consider the case where a client is synchronizing with more than
> one server (all our PRO clients have multiple profiles). For that,
> even without suspend&resume, you need a mechanism to pinpoint changes
> on a time scale at multiple points, even more than 2 (if you have more
> than 2 servers). That what the binfileXXX layer was created for in the
> first place.
>
> 2) Even in a single server scenario resume, you must be able to
> differentiate modifications happened after the original sync and
> before the suspend and modifications happened AFTER the suspend. You
> are right that this differentiation does not necessarily need a re-
> evaluation of the changes since last regular sync as long as the
> intermediate layer keeps a list of to-be-resent items somehow.
>
> Best Regards,
>
> Lukas Zeller ([email protected])
> -
> Synthesis AG, SyncML Solutions & Sustainable Software Concepts
> [email protected], http://www.synthesis.ch
>
>
>
--
Regards,
Chen Congwu
Moblin China Development
--- End Message ---
_______________________________________________
os-libsynthesis mailing list
[email protected]
http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
