You should be able to use a single on_change rule for all the DBs if you give the DBs a name that can be parsed, eg user_<username> as you can use a regex in the on_change rule. Spiegel will automatically take care of creating change_listener docs to keep track of the place in the _changes feed. You can completely tune concurrency and batch sizes and Spiegel will only use resources when there is actually a change.
An alternative to Spiegel is to listen to the global changes feed, etc... yourself but believe me when I say this takes a lot of work, especially when you need to make your design scale :) On Tue, Nov 28, 2017 at 9:37 AM Sachin Divekar <[email protected]> wrote: > That should work, I think. But do Spiegel change listeners required for all > the databases to achieve this? > > Anyway, I am going to try all the solutions discussed in the last post and > this one to get a better understanding. > > Regards > Sachin > > On Tue, Nov 28, 2017 at 8:53 PM, Geoffrey Cox <[email protected]> wrote: > > > There is also a variation on 3. (the custom API) where you do use > PouchDB. > > You could have users write messages to their database and then use > Spiegel > > with an on_change rule > > <https://github.com/redgeoff/spiegel/blob/master/DESIGN.md#on_change> > that > > responds to new/edited messages and then copies those messages to the > > target database(s). > > > > On Tue, Nov 28, 2017 at 7:00 AM Geoffrey Cox <[email protected]> wrote: > > > > > Hi Sachin, > > > > > > I work on a teaching app called Quizster that handles some of the same > > > concerns. Here is my 2 cents: > > > > > > On Tue, Nov 28, 2017 at 6:37 AM Sachin Divekar <[email protected]> > wrote: > > > > > >> Hi, > > >> > > >> I have been studying a possibility to use Couchdb for a realtime > > messaging > > >> app. The architecture I am thinking about is as follows. The app will > > >> access only the local offline database using Pouchdb which is in > > >> continuous > > >> replication with a remote Couchdb cluster. On the server, it'd be one > > >> database per user. The app, in addition to conversations also has > other > > >> active objects like tasks, notes etc. I just want to have a discussion > > >> with > > >> some experienced Couchdb users if such architecture makes sense. > > >> > > >> My primary concerns are following. > > >> > > >> 1. I am thinking of keeping shared objects in every participant user's > > >> database. e.g. a group chat will be present in each of the > participant's > > >> database or a task can have multiple users assigned to it. I am not > sure > > >> how to replicate the changes made by one user on a task in his > database > > to > > >> other participant's database. Perhaps using filtered replication but I > > am > > >> not sure. > > >> > > > GC: I think you have a few options here: > > > 1. You can use filtered replication to replicate the docs, where you'd > > > probably need a replication for each pair of users who are sharing the > > docs > > > to replicate to and from the respective user's DBs. > > > 2. If you can identify a common group, e.g. in our app we have classes, > > > and don't need this group to write to these common docs then you can > use > > a > > > database to replicate docs from a user DB to this "group" database. > Users > > > could then read from this group DB. Perhaps in your case you can think > > of a > > > group like a Slack organization or channel. You can then use a role to > > > restrict read access to this group DB. You'd still need a replication > per > > > user-group, but you'd probably need a lot less of them. > > > 3. You could create a custom API, instead of writing data with PouchDB, > > > (just use PouchDB for reading) that you call when the user sends a > > message > > > that saves the message to all the needed databases. This has the > distinct > > > disadvantage that without some serious work on your side that you app > > > wouldn't work offline. > > > > > >> > > >> 2. Will the load of continuous replication of a few thousand databases > > be > > >> handled smoothly? > > >> > > > GC: This is great question and this load is definitely an issue. The > good > > > news is that we just open sourced our solution for this: Spiegel > > > <https://github.com/redgeoff/spiegel> > > > > > >> > > >> 3. How to deploy filters, views etc on all the databases because > > >> essentially they will be same for all the databases. > > >> > > > GC: We create migration scripts that use Slouch StreamIterators > > > < > https://github.com/redgeoff/slouch/wiki/Don%27t-just-relax.-Slouch%21# > > streamiterators> > > > to iterate through all the target databases and create the filters and > > > views. > > > > > >> > > >> Looking for some advice. > > >> > > >> Thank you. > > >> > > >> Regards > > >> Sachin > > >> > > > > > >
