[
https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860127#comment-15860127
]
Chris Douglas commented on HDFS-11026:
--------------------------------------
bq. If the long (I know, I said byte) is a magic number like min or max long,
decode rest as PB which will include the real expiry, else continue decoding as
writable.
In #2, the magic number matches the {{field number << 3 | type}} from PB (0x08
in the current patch, IIRC). Using the {{WritableUtils}} parsing machinery is a
nice touch, though we still need the marker byte at the head of the stream when
we call protobuf. We're still prevented from removing the expiryTime field
while we support 3.x. I suppose it can evolve to #3, where we include the
expiryTime field to support 3.x clients. Implementors not targeting 2.x get a
normal PB record.
What you describe is closer to #3, where the first byte is an optional PB field
that we strip away before handing the record to protobuf. The protobuf parser
doesn't know or care if we strip the first, optional field. It'll be vestigial
in future implementations that don't support backwards compatibility with 2.x.
Parsing is simplified because we don't need mark/reset (or similar), but we add
a (trivial) overhead to each record and a weird compatibility case. Also the
indignity of adding a version field to a DDL whose purpose was to manage
versioning.
Again, we're splitting hairs, here. It's very unlikely we'll remove/replace
expiryTime, or notice the overhead of an explicit version field. I'm +1 on the
current patch (with a note in the .proto to help future maintainers), but would
be fine with either variant.
> Convert BlockTokenIdentifier to use Protobuf
> --------------------------------------------
>
> Key: HDFS-11026
> URL: https://issues.apache.org/jira/browse/HDFS-11026
> Project: Hadoop HDFS
> Issue Type: Task
> Components: hdfs, hdfs-client
> Affects Versions: 2.9.0, 3.0.0-alpha1
> Reporter: Ewan Higgs
> Assignee: Ewan Higgs
> Fix For: 3.0.0-alpha3
>
> Attachments: blocktokenidentifier-protobuf.patch,
> HDFS-11026.002.patch, HDFS-11026.003.patch, HDFS-11026.004.patch,
> HDFS-11026.005.patch
>
>
> {{BlockTokenIdentifier}} currently uses a {{DataInput}}/{{DataOutput}}
> (basically a {{byte[]}}) and manual serialization to get data into and out of
> the encrypted buffer (in {{BlockKeyProto}}). Other TokenIdentifiers (e.g.
> {{ContainerTokenIdentifier}}, {{AMRMTokenIdentifier}}) use Protobuf. The
> {{BlockTokenIdenfitier}} should use Protobuf as well so it can be expanded
> more easily and will be consistent with the rest of the system.
> NB: Release of this will require a version update since 2.8.x won't be able
> to decipher {{BlockKeyProto.keyBytes}} from 2.8.y.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
