Hi,
I have a few questions about the handling of the log buffer.
When we can't write anymore data in the buffer, because it's full, we
try to flush the buffer on disk. What happens then is :
- if there is enough room remaining in the buffer, we write a skip
record (with a -1 length) : is it necessary ? (we then rewind the buffer)
- otherwise, we rewind the buffer
In any case, we increment the writeAheadRewindCount : what for ?
then we call the flush() method, which will be executed only if there is
no other thread flushing the buffer already (just in case the sync()
method is called by another thread). I guess this is intended to allow a
thread to add new data in the buffer while another thread writes the
buffer on disk?
So AFAIU, only one thread will be allowed to write data into the buffer,
up to the point it reaches a record being hold by the flush thread, and
only one thread can flush the data, up to the point it reaches the last
record it can write (which is computed before the flush() method is called).
I'm wondering if we couldn't use a simpler algorithm, where we have a
flush thread used to flush the data in any case. If the buffer is full,
we stop writing until we are signaled that there is some room left (and
this is the flush thread role to signal the writer that it can start
again). That means we write as much as we can, signaling each record to
the flush thread, and the flush thread will consume the record when they
arrive. If both are colliding (ie, no more room remains in the buffer,
the reader will have to wait for the writer to wake it up). We won't
need to use a buffer at all, we just pass the records (plus their
headers and trailers) in queue, avoiding a copy in a temporary memory.
This is basically doing the same thing, but we don't wait until the
buffer is full to wake up the writer. This is the way the network layer
works in NIO, with a selector signaling the writer thread when it's
ready to accept some more data to be written.
thougths ?
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
