If you haven't seen it on HN [0], Hyperdex just dropped a leveldb fork they are calling HyperLevelDB [1]. It seems like they are looking for drop-in replacement compatibility. I'm interested to hear thoughts from the Basho folks (or anyone else) once they've had a chance to look it over. One of their specific goals is maintaining performance during compaction which I know has been an issue for some people using the leveldb backend.
If you read their post there seems to be three things they are doing to reach their higher performance: 1. multi-threaded synchronized write ordering. "LevelDB uses very coarse-grained synchronization which forces all writes to proceed in an ordered, first-come-first-served fashion, effectively reducing throughput to that of a single thread. HyperLevelDB increases concurrency by allowing multiple threads to agree on the order of their respective writes, and then independently apply the writes in a manner consistent with the agreed-upon order." 2. tuned write delay on compaction. "HyperLevelDB removes this artificial delay, allowing the application (in our case, HyperDex) to independently decide to delay writes, using information available outside the scope of LevelDB." 3. tuned intra-level re-writes. "LevelDB's compaction algorithm is not efficient, and in the "fillrand" benchmark will, on average, rewrite 3MB of data in the upper level for every 1MB of data in the lower level. HyperLevelDB avoids this waste by selecting the compaction with the smallest overhead." Thanks, -Alexander Sicular @siculars [0] https://news.ycombinator.com/item?id=5833414 [1] http://hyperdex.org/performance/leveldb/.
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
