On 03/02/2010 01:31 PM, Pete Zaitcev wrote:
This patch adds the self-check to Chunk. With it, the daemon can rescan
all of its keys and drop those that fail to match their own checksums
or throw an I/O error. Objects that are found found faulty are made
invisible to applications (back-end files are renamed, so that bad
blocks are not reused). This is intended to work in concert with
applications that store redundant copies of their data objects.
This patch includes the part that tracks the active I/O, since the
self-check is the only user of it, so it made little sense to separate
the two. We have to track the I/O so that self-check does not mistakenly
assume a partially stored object to be faulty and kills it.
Running the self-check can adversely affect performance. As a crude
way to limit the problem, we limit the load to one check thread only.
Still, as anyone who had mlocate or Beagle running in their desktop
knows, the biggest issue is not additional I/O as such, but the blowing
away the page cache and dentries in kernel. Also, our scheduling is
between rudimentary and non-existing. We only provide a looping check
with a randomized delay and an external control to start and verify the
running of the checking.
Therefore, to avoid surprises with sudden loss of objects and with
performance anomalies, periodic self-check defaults to off.
Still, self-check is an intergral part of the daemon, so we include
unit tests for both the I/O tracking and self-check itself.
Signed-off-by: Pete Zaitcev<[email protected]>
---
doc/setup.txt | 10 +
include/Makefile.am | 2
include/chunk-private.h | 4
include/chunk_msg.h | 21 ++
include/chunkc.h | 4
include/objcache.h | 75 ++++++++
lib/chunkdc.c | 82 +++++++++
server/Makefile.am | 3
server/be-fs.c | 130 ++++++++++++++
server/chunkd.h | 28 +++
server/cldu.c | 1
server/config.c | 18 ++
server/objcache.c | 138 +++++++++++++++
server/object.c | 9 +
server/selfcheck.c | 293 +++++++++++++++++++++++++++++++++
server/server.c | 135 +++++++++++++++
test/.gitignore | 2
test/Makefile.am | 7
test/objcache-unit.c | 64 +++++++
test/selfcheck-unit.c | 334 ++++++++++++++++++++++++++++++++++++++
test/test.h | 2
tools/chcli.c | 116 +++++++++++--
22 files changed, 1454 insertions(+), 24 deletions(-)
applied... looks mostly OK. Things like self-check period still need
to be removed, and a few other minor quibbles. But let's go ahead and
get this in, then tackle those things.
--
To unsubscribe from this list: send the line "unsubscribe hail-devel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
