On Wed, May 7, 2008 at 2:19 PM, Brian Smith <[EMAIL PROTECTED]> wrote:
> > Mark Stahl wrote: > > > One issue that we've encountered with tombtones is that most services > > don't > > keep them forever. (Otherwise, the list of tombstones only ever > > grows.) > > > > For most systems, keeping track of deleted items on the back end is not a > problem. The only problem is that you don't want the feeds to be bloated > with an ever-increasing number of tombstones, right? Typically, the limitation on the number of tombstones reflects a limitation in the amount of storage available in a backend service. If tombstone lifetimes are infinite, the storage required for an atom pub collection can grow to an ever increasing size, even if the number of elements allowed in that collection is bounded. Most services I talk about doing AtomPub purge tombstones after some time (for example, 30 days.) So I'm faced with the problem today. Advertising the tombstone window would solve a lot of problems. > This leads to problems with clients that periodically poll for changes - > they don't know if the set of tombstones is complete since the last time > they polled. > I'd suggest adding an element at:deleted-since, so the server can let the > client know the tomebstone window in effect. > To solve this problem, I implemented a simple feed paging extension. The > first page of the collection feed (at the collection URI) contains a > "feed-changes" link. The feed-changes link points to a feed that is only > guaranteed to contain all the information (e.g. entries and tombstones) that > was changed since the first page of the collection feed was fetched. The > clients can use this to answer the question "what are all the changes that > occurred since the last time I synced with the server?" > > The first page of the "feed-changes" feed contains another "feed-changes" > link that works in an analogous fashion. Clients can keep in sync with the > collection by chasing the "feed-changes" links. The "feed-changes" feeds can > also use RFC 5005 paging; to get a complete sync, one has to page through > all the pages (using "next") before chasing the "feed-changes" link from the > first page. This addresses the large feed document issue, but is still premised on the idea that you store all tombstones indefinitely, so it doesn't solve the problem I'm attempting to solve. My suggestion is to finish up the at:deleted-entry mechanism that is as > simple as possible, and then define a separate AtomPub synchronization > protocol on top of it, based on the "feed-changes" mechanism I just > described, where the at:deleted-entry mechanism would be a required element > of the "feed-changes" feeds. What do you think? > I'm willing to separate the problem of a sync spec from tombstones, so I guess it depends on whether you consider it important that the tombstone spec have the capability to indicate that the set of tombstones provided may not be complete. I tend towards thinking that it's important enough to be included in the main spec, but YMMV, and I'd be happy to have at:deleted-entry today so we can build on it. > Another useful piece of information to the client trying to sync is a > > hint > > as to the typical tomebstone lifetime. But I'm not sure if it belongs > > in > > this spec. (Perhaps it's more appropriate in a service document?) > > > It is a good idea to keep this information in the feed, because this is > useful in Atom feeds that are not AtomPub feeds. The reason I hesitate to mention this is it seems to be heading more and more towards a sync spec. And I'm not trying to solve the whole sync problem with this spec, but create tools that will be used to solve sync. Regards, > Brian > -- Test driving http://five.sentenc.es/
