The following is a couple of questions (and suggestions) regarding the
technical sides of Riak performance and reliability.
The questions have been prompted by reading Riak source code and by
discussions within our company.
I suppose the common thread here is "latency hiding"...
Durable Writes.
---------------
The default for bitcask's 'sync_strategy' setting is not to flush to
disk explicitly at all.
This means that a 'Durable Write' isn't actually durable; the difference
between 'W' and 'DW' replies is whether the write has made it past Riak,
to the OS - but not through the OS and down to disk.
Is this correct?
What I'd have expected, as a reaonably-performing alternative, is that
Riak would flush periodically - say, after at most W_flush writes or
MS_flush milliseconds, and send 'dw' replies for all of the relevant
requests (those written since last flush) at once after the flush has
been completed.
This would combine 'real' DW semantics with reasonable performance (and
is how I have handled a similar problem; my conceptions about what is
right and proper may of course be influenced by my own coding history...).
(For kicks, MS_flush might even be dynamically determined by how long
the flush operations tend to take; the typical value of a flush
duration, multiplied by a small constant, would probably be a fitting
value.)
Parallel Reads.
---------------
Within a vnode, bitcask read operations happen in serial.
Is there any reason for reads not happening in parallel?
For map/reduce operations, in particular, I imagine this might make a
difference, by giving the OS the opportunity to schedule disk accesses
so as to reduce seek time.
(Unless of course Riak itself reorders the keys before reading, but I
don't believe this is the case - especially since the order would depend
on the backend: for bitcask, by time; for innostore, by key order, for
instance.)
Of course, if each host has multiple vnodes, there will be some
parallellity even with serialized reads within each bitcask.
Regards,
Erik Søe Sørensen
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
