Ok, another test.. using the paging forward in time approach (e.g. starting from the last page I looked at, GET each subsequent "next" page containing entries modified since the last run as you describe below... running 100 iterations pulling the feeds from localhost, with pipelining enabled, the times for each run (in ms) are (first run is thrown out).
400 entries over 20 pages: [520, 364, 593, 1092, 547, 971, 628, 769, 368, 1557, 462, 293, 495, 889, 991, 457, 559, 1040, 391, 309, 296, 325, 275, 552, 397, 303, 308, 341, 360, 299, 430, 322, 308, 321, 275, 314, 318, 332, 321, 431, 400, 488, 322, 343, 271, 369, 387, 404, 324, 464, 436, 392, 392, 1108, 320, 537, 275, 330, 302, 277, 403, 348, 265, 275, 364, 637, 351, 293, 276, 268, 275, 273, 301, 298, 557, 304, 360, 468, 368, 435, 375, 354, 331, 372, 417, 335, 485, 315, 356, 292, 288, 296, 320, 292, 384, 312, 416, 257] 400 entries in a single feed: [204, 482, 251, 191, 577, 163, 150, 186, 223, 136, 159, 168, 170, 223, 905, 186, 166, 139, 157, 145, 283, 143, 150, 397, 144, 146, 185, 167, 313, 177, 147, 177, 143, 462, 162, 158, 136, 137, 162, 115, 265, 121, 183, 319, 121, 405, 169, 545, 163, 191, 156, 535, 146, 150, 518, 118, 144, 263, 145, 152, 132, 152, 509, 123, 165, 130, 171, 139, 135, 122, 152, 121, 128, 147, 117, 168, 141, 144, 142, 464, 578, 142, 143, 139, 143, 124, 151, 181, 214, 117, 160, 176, 134, 150, 116, 150, 150, 323, 127] - James Brian Smith wrote: > James M Snell wrote: >> Either way, it's generally the same amount of data being >> transferred, only you have to deal with multiple requests and >> scanning through each page to detect which items have >> changed. Even with pipelining, that's less efficient than >> making a single request with a single response that you know >> for certain contains only items that have been modified. > >> So long as the feeds contain only the entries that were >> modified within the specified period of time then yes, >> without paging the approaches are equivalent. In other >> words, I want to avoid having to scan through the entries to >> determine which entries have or have not changed since the >> last time I looked at the feed; instead, I want to know that >> when I'm syncing, ALL of the entries listed in that feed >> document have changed since the last time I sync'd. > > The subscription page of a collection feed does not usually have a > "next" link, and the only way to find new entries in a collection > is to poll the subscription page of the collection feed (at the > collection URI). My suggestion is: (a) the server adds a "next" link > to the subscription page, that links to a document that represents all > entries nwerer than the currently-newest entry, (b) the client chases > these "next" links and thus never has to poll the collection URI. By > doing this, the client will never encounter an entry that it already has > an up-to-date copy of. > > - Brian > >
