On Tue, Nov 28, 2017 at 8:30 PM, Geoffrey Cox <[email protected]> wrote:
> Hi Sachin, > > I work on a teaching app called Quizster that handles some of the same > concerns. Here is my 2 cents: > > Quizster: wow, that's one awesome idea. On Tue, Nov 28, 2017 at 6:37 AM Sachin Divekar <[email protected]> wrote: > > > Hi, > > > > I have been studying a possibility to use Couchdb for a realtime > messaging > > app. The architecture I am thinking about is as follows. The app will > > access only the local offline database using Pouchdb which is in > continuous > > replication with a remote Couchdb cluster. On the server, it'd be one > > database per user. The app, in addition to conversations also has other > > active objects like tasks, notes etc. I just want to have a discussion > with > > some experienced Couchdb users if such architecture makes sense. > > > > My primary concerns are following. > > > > 1. I am thinking of keeping shared objects in every participant user's > > database. e.g. a group chat will be present in each of the participant's > > database or a task can have multiple users assigned to it. I am not sure > > how to replicate the changes made by one user on a task in his database > to > > other participant's database. Perhaps using filtered replication but I am > > not sure. > > > GC: I think you have a few options here: > 1. You can use filtered replication to replicate the docs, where you'd > probably need a replication for each pair of users who are sharing the docs > to replicate to and from the respective user's DBs. > That is the exact approach I could think of. However, in that case I will need a facility to set the target dynamically in the replicator document. I haven't found any solution for that. Also, each replication job from _replicator database needs to open a connection between source and target database, if I have understood it correctly. In that case in my schema I will have a mesh of connections e.g. for a calendar event object, there will be connectoins between databases of all the participants or in case of a group communication they will be between all the group members. Or another potential solution that occurred to me is subscribing to change feed and inserting/updating the shared document in all the user's databases manually, but again the issue is that I need to have as many listeners as the number of users which is a waste of resources. That lead me to another possible solution of using external API handler ( http://docs.couchdb.org/en/2.1.1/externals.html) to copy the objects -- I am not sure if I've understood the external API, handler and OS daemon matter at all. I am planning to write a small handler to get some understanding. I checked Spiegel's documentation. Can its change listening feature help in some way? 2. If you can identify a common group, e.g. in our app we have classes, and > don't need this group to write to these common docs then you can use a > database to replicate docs from a user DB to this "group" database. Users > could then read from this group DB. Perhaps in your case you can think of a > group like a Slack organization or channel. You can then use a role to > restrict read access to this group DB. You'd still need a replication per > user-group, but you'd probably need a lot less of them. > That may work, thanks. Let me read on role based access control. If I can somehow restrict/allow users from reading/writing certain documents in a database, that will solve this problem. 3. You could create a custom API, instead of writing data with PouchDB, > (just use PouchDB for reading) that you call when the user sends a message > that saves the message to all the needed databases. This has the distinct > disadvantage that without some serious work on your side that you app > wouldn't work offline. in that I can have a routine > The reason for choosing CouchDB+PouchDB based architecture is to build an offline first app which will only talk to the local DB so that to begin with we don't need to care about building custom APIs -- CouchDB will take care of syncing data which simplifies the overall architecture significantly. > > > 2. Will the load of continuous replication of a few thousand databases be > > handled smoothly? > > > GC: This is great question and this load is definitely an issue. The good > news is that we just open sourced our solution for this: Spiegel > <https://github.com/redgeoff/spiegel> > > I can see that Speigel's smart approach to replication can help in scaling, but does it increases latency because there won't be already open connection? Also, with CouchDB 2.x does the clustering facility and shards etc. improve scalability and latency? Does Spiegel play nicely with a clustered CouchDB? > > > 3. How to deploy filters, views etc on all the databases because > > essentially they will be same for all the databases. > > > GC: We create migration scripts that use Slouch StreamIterators > <https://github.com/redgeoff/slouch/wiki/Don%27t-just-relax.-Slouch%21# > streamiterators> > to iterate through all the target databases and create the filters and > views. > > Super. We had already made our minds to write a tool to perform such activities. With Slouch things can become super easy to implement. Is something like following possible? A template database with all the generic design docs which is getting replicated to all the user databases? Thank you. Regards Sachin
