Let's not conflate two things here. The changes feed is just the documents in the database, including deleted ones, in the order they were last written. This is simple to do in FoundationDB and Adam already showed the solution using fdb's versionstamps.
The talk of 'watcher' and 'subscriber' is about the continuous mode of changes feed and this is a stateful part of the current couchdb system (as it is mediating the writes to the db). We know the watcher facility in fdb will not scale to this usage. B. -- Robert Samuel Newson rnew...@apache.org On Mon, 4 Feb 2019, at 19:18, Ilya Khlopotov wrote: > One of the features of CouchDB, which doesn't map cleanly into > FoudationDB is changes feed. The essence of the feature is: > - Subscriber of the feed wants to receive notifications when database is > updated. > - The notification includes update_seq for the database and list of > changes which happen at that time. > - The change itself includes docid and rev. > Hi, > > There are multiple ways to easily solve this problem. Designing a > scalable way to do it is way harder. > > There are at least two parts to this problem: > - how to structure secondary indexes so we can provide what we need in > notification event > - how to notify CouchDB about new updates > > For the second part of the problem we could setup a watcher on one of > the keys we have to update on every transaction. For example the key > which tracks the database_size or key which tracks the number of > documents or we can add our own key. The problem is at some point we > would hit a capacity limit for atomic updates of a single key > (FoundationDB doesn't redistribute the load among servers on per key > basis). In such case we would have to distribute the counter among > multiple keys to allow FoundationDB to split the hot range. Therefore, > we would have to setup multiple watches. FoundationDB has a limit on the > number of watches the client can setup (100000 watches). So we need to > keep in mind this number when designing the feature. > > The single key update rate problem is very theoretical and we might > ignore it for the PoC version. Then we can measure the impact and change > design accordingly. The reason I decided to bring it up is to see maybe > someone has a simple solution to avoid the bottleneck. > > best regards, > iilyak
