Toru Maesaka: BlitzDB Concurrent Testing and Write Performance
Last month while being at the MySQL Conference, several people asked me about the status of BlitzDB. Specifically, they were interested in when I’ll release BlitzDB. Fair enough – I’ve been working on this project long enough for people to start questioning this.
The answer is, BlitzDB is done in terms of implementing the design. Right now it’s about finding bugs, fixing it and testing BlitzDB’s stability under concurrent load. Thanks to the motivation boost I gained at the conference, I’ve now fixed the bugs that were slowing me down and I’m gradually adding more tests into BlitzDB’s test suite. I consider BlitzDB’s initial release to be the day it gets merged into Drizzle’s trunk. This is almost ready as BlitzDB seems to be building fine on Drizzle’s Build Farm infrastructure. However, I won’t move to the next step until I’m satisfied with BlitzDB’s stability.
Yesterday I spent some time doing some concurrency testing on BlitzDB’s INSERT code with skyload. Needless to say, concurrency testing is also a convenient way to look at the performance of a particular component. So, I decided to publish my findings from this test. First, here is the background of the test.
Purpose of the Test
- Test BlitzDB’s slot-lock mechanism.
- Confirm that BlitzDB will not crash under concurrent INSERT workload.
- Confirm that key insertion to the index is working as expected.
- Confirm that writes to multiple indexes work as expected.
- Observe the write-performance impact of adding an index.
Two commodity boxes were used. One dedicated for the client and the other dedicated for the server (Drizzle + BlitzDB). Both boxes has the same spec: Intel Quad Xeon E5345 (2×4MB L2 cache), 8GB Memory, 500GB SATA II, gigabit NIC. Servers were connected by a gigabit switch. File system on the server was ext3.
By default, a BlitzDB table is optimized for up to 1 million rows. Therefore this test inserted 1 million rows to a table with different concurrency levels. A different concurrency level is used per run. The table used in this test only contains three integer columns. Tests are performed up to three indexes. The linux kenel’s dirty buffer is flushed before each test run. Tests were run until the performance curve flattened.
Result
As seen above, scalability from 1 thread to 4 thread showed an ideal curve. This is expected since the server is a 4 core box. From 4 threads, performance showed some improvements up to 12 threads. From there on, concurrency greatly exceeds the number of physical cores so we can’t observe decent performance growth. The highest insert QPS gained in this test was just over 86,000 QPS. With more cores on the server and more clients, I suspect BlitzDB can hit over 100k QPS.
Although this graph looks good at first sight, I’m not happy with it. The performance penalty for adding multiple indexes should be greater than what’s observed in this result. This is because TC’s B+Tree is internally protected by a single lock on writes. I suspect that the performance penalty is not observed in this graph because I didn’t give BlitzDB enough load to make TC work hard. This implies that a bottleneck could exist elsewhere (Network, Drizzle or BlitzDB’s handler level code).
However, I’m glad that BlitzDB stood stable on this concurrency test which was what I wanted to test in the first place. Admittedly I need to mix several types of queries to properly test BlitzDB’s stability. I plan on doing this next with sysbench and hopefully RQG.
Once this is done, I’ll submit a merge proposal to the Drizzle Project :)
Future Development Plans
- Find bugs, Fix bugs, Repeat.
- Write an inbuilt auto recovery routine.
- Eventually add a crash safe option to BlitzDB.
URL: http://torum.net/2010/05/blitzdb-concurrency-testing/
_______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
