Peter Somogyi back almost three years ago proposed the following protocol enhancement for creating dump files containing compressed data. An implementation of this proposal is available for review in OpenAFS RT Ticket 17947.
Feedback is requested.
Jeffrey Altman
Compression support for OpenAFS - specification
- additional param for "volserver" command line:
-z [gzip:<level>|bz2:<level>]
where level: [1..9|default]
Example:
/usr/afs/bin/volserver -z gzip:5
- new volserver must detect "not to be compressed" new flag in volume header:
- VolumeDiskData.reserved1[0] & 1
- new "vos" setfields flag:
- cmd line: compression {enabled,disabled}
- "use" volintInfo.spare[2]&1 and volintInfo.spare[3]&1
- modify VolSetInfo and ListOneVolume
- modify "vos examine" to show this flag
- new volserver must provide 1 new RPC call in the interface:
- RXAFS_GetCapabilities - same as on fs interface
- 2 separate bits for zlib and bz2 support
- new volserver must query this function (if exists) in
VolForwardMultiple and VolForward
- VolForwardMultiple: send compressed format only when ALL receivers
support that
- new volserver must produce always compressed dump if the compression is
turned on - at "vos dump"
- new dump format:
volserver dumps:
<0x01><DumpHeader>
<0x02><VolumeHeader>
<0x03><VNode1>
[<0x06><length ::= 4><afs_uint32: flags]
<0x05><unknown length::=0x80><CompressData>
<0x04><DumpEndMagic>
[EOF]
where:
- <CompressData> ::= <CompressHeader><CompressContent>
- <CompressHeader> ::= <length><byte: CompressionType> //where
length is sizeof(<CompressHeader>)-sizeof(<length>)
- <CompressContent> ::= the zlib/bzip2 generated content
- <VNode1> must be the first large vnode (error case if it
doesn't exist):
<VNode1> ::= <vnodeNumber><uniquifier>{<tag greater
than 20><contents>}*
- <length>: variable size length, 0x80=unknown (means that this
section must be parsed to proceed)
- If bit 7 is not set, bits 0-6 are the length
- If bit 7 is set, bits 0-6 tell you how many bytes
long the length is.
The length follows immediately in MSB-first order.
- The special value 0x80 means indefinite length.
- 0xfe and 0xff would indicate a single-bit value, with
the value stored in the low-order bit of the length.
- after <0x06>, "extra flags" must be followed (tag with
explicit length), containing one or more afs_uint32's
(the first afs_uint32 will be transmitted into
VolumeDiskData.reserved1[0] - for example the DONT_COMPRESS ::=0x01 flag)
If there's no <0x06> tag, the volume's compression must be
enabled.
volserver parser:
- unknown high-level tags treatment:
- <0x06..0x60>: <length> awaited, and then <length>
bytes must be skipped, write warning to VolserLog
- <0x61..0x7a>: skip 4 bytes, write warning to VolserLog
- the tag <0x7e> has no data by itself, but labels the
next tag as "critical"
- all of 0x7b-0x7f will be single-byte tags with no
data (skip them if they're not "critical", otherwise error)
- 0x80-0xff: error
- <CompressData> mustn't appear more times in the dump. (It
would cause more complex buffering implementation.)
- no <CompressData> embedding supported
- include zlib and bzip2 (src/zlib and src/bzip2)
- "don't compress" flag must be transmitted only when the source volserver's
compression is set, and each receiver volserver supports this flag (in case of
"vos release"), or when "vos dump" was invoked
Peter Somogyi
2005-FEB-04
[EMAIL PROTECTED]
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ AFS3-standardization mailing list [email protected] http://michigan-openafs-lists.central.org/mailman/listinfo/afs3-standardization
