@Scott, thanks for your comment, but I think we are talking about different scenarios and different speed levels :)
??? Not on any real RDBMS like MySQL, PostgreSQL... If you want reported message to be durable, you need either to replicate it to another server, or write it to disk. In-memory log doesn't help here, since transaction is still flushed on commit. You can tune up your DB, for example, switching off committing after each transaction (InnoDB in MySQL has this possibility, as far as I remember), but even in this case I doubt MySQL or PostgreSQL will be able to beat 10K/sec. With ZMQ I've got 300K/sec from the very first time on my dev machine. > Why would being a key-value store make getting results slow? > I think really all depends on the indices that you build... > Indices also have their cost. And, most important, they are not really needed here. In Kafka, for example, new records are simply added to the end of the queue, and pointer to the end is moved further. In our tests it gives about 50K/sec, which is much higher than for RBMSs mentioned above, and unlike most key-value storages provides easy and fast way to read all messages down the stream.
