On Feb 15, 2013, at 1:41 AM, David Barbour <dmbarb...@gmail.com> wrote: > > A technique I've seen used (albeit, only once or twice) is to batch updates > from multiple connections, and run batches at 20+Hz while there is demand for > them. This only requires one connection from the server to its database, and > allows you to respond and close each HTTP connection in a predictable amount > of time, and allows you to ensure the updates are committed before replying. > The limitation of this technique is that you can't have ad-hoc updates, since > those would be too difficult to batch consistently and sensibly. > > It's a bit specialized, but is able to handle a lot of updates.
Distributed database engines that do this natively are becoming popular. They are usually referred to as "short transaction processing" databases and workloads. By forcing all open transactions to commit or rollback several times a second it is possible to guarantee strict serializability in a distributed database if you can package your transactions within that window. There are a few variations on the idea. Traditional transaction processing databases allowed transactions to be open for an unbounded period of time. There are stories passed around among DBAs of zombie transactions that had been open for months or years. For large-scale distributed databases, the cost of coordinating the transaction state is prohibitive and limits the total transaction throughput. Some clever person noticed that if you can strictly bound the transaction length then it makes the distributed transaction problem much simpler. Conveniently, many transaction processing applications can be made to fit within the STP database model. An STP database engine can support orders of magnitude more transaction throughput than a conventional OLTP database. The tradeoff is that the transaction latency is higher. 10-100 milliseconds seems to be the zone where latency is acceptable but you can still fit most transactions into that window. _______________________________________________ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
