This patchset is just about ready to go upstream. Just need to write a
couple tests (familiar refrain eh?:)).
These changes add a new Get-Partial-Object (GET_PART) chunkd operation.
GET_PART permits partial retrieval of an object, by adding an
(offset,length) pair to the standard Get-Object (GET) operation.
length==0 is special-cased as meaning retrieve until end of object.
The maximum number of bytes that may be requested in a single GET_PART
request is 4 x 64k blocks (256k). Larger lengths will be truncated
down to the maximum.
Because we currently only store whole-object SHA1 checksums, we are left
without an ability to verify on-disk data is valid, when retrieving a
subset of an object. Thus, a necessary pre-req of GET_PART is changing
the checksum scheme, which is done as follows:
* objects are defined as runs of 64k logical blocks
* checksums are stored on-disk for each 64k in an object
* Rather than returning the stored SHA1 checksum, which serves
to verify both on-disk and network integrity, we break this
into two steps,
* verify per-64k checksums at GET_PART time
* generate on-the-fly SHA1 checksum for GET_PART
returned data
The chunkd network protocol supports any offset/length, including
not-64k-aligned values. However, failure to align GET_PART requests on
64k boundaries will result in reduced performance, due to additional
work chunkd must perform [and then throw away], because chunkd now works
in 64k chunks internally.
This is a major protocol milestone, and should immediately enable sane
usage by nfs4d and itd (see wiki if unfamiliar), as well as hopefully
providing useful benefits to tabled as well.
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
