Nikolai You would have a rapidly growing database on your nodes as well because each document update on each node would generate a new version.
I offer this:- Have a document in your master database with the URLs of the nodes and have a cron job to read that document and initiate a one time replication with each node and compress your "master" database (you can use a shell script and curl to do this). Now, you will need to either multi thread the replication cron script or in some way test that the host is available before trying the replication because if the remote host is not available your script will stall for a good few seconds when trying to contact the remote host and if you have a lot of hosts and a short(ish) cron cycle you can get cron job collision. Have the status collection job on each node compress the database on each update. Bob On 14 Mar 2011, at 12:41, Nikolai Teofilov wrote: > Hi all, > > I need a help on a concept for a monitoring system based on CouchDB. I have > master server with a database "status" and several, around 20 nodes machines > (the number of machines will grow) performing some logging on a local > instances of CouchDB. Each local machine (node) has a database "status" as > well and logs realtime data in a single document named as the computer. The > idea is to monitor all the data from all node machines in a single database > on the master server. The final result should be a database that is a > collection of documents named as the computer names of all node machines and > each document reflect the current status of the node. Then it will be easy to > have long-poll handler for monitoring and rendering the changes of all > machines on the master server as a status panel... > > I made a local db "status" on each node machine and started a continuous > replication to the master server "status" database. The problem is that some > of the logging processes on the node machines are working over hours and can > involve logs each second. This will reflect on the size of the master > database. How to handle this correctly? I can compact the master target > database from time to time remotely. Is this a good idea or is there any > other way to make continuous replication with kind of auto compact feature? > Would a compacting from time to time will affect somehow the other continuous > replications? > > The second problem I have is once I delete (recreate) the source local > "status" database the old instance of the continuous replication does not > work for the new created database. I have a kind of workaround be canceling > the old continuous replication and triggered it again. This work well but I > am not sure if I can expect some other problems somehow. > > Any suggestions or thoughts? > > Thanks > Nikolai
